[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:58:59 dist-test-slave-tm1g krb5kdc[25733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:58:59 dist-test-slave-tm1g krb5kdc[25733](info): set up 2 sockets
Aug 06 15:58:59 dist-test-slave-tm1g krb5kdc[25733](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 15:59:00.658659 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 15:59:00 dist-test-slave-tm1g krb5kdc[25733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:59:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 15:59:00.777755 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42313
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:42313
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:00.887696 25747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:00.887988 25747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:00.888064 25747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:00.892041 25747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:00.892130 25747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:00.892169 25747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:00.892206 25747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:00.892243 25747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:00.897660 25747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:42313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:00.898754 25747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:00.899775 25747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:00.908566 25753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:00.908625 25755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:00.908694 25747 server_base.cc:1034] running on GCE node
W20260806 15:59:00.908635 25752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:00.909365 25747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:00.910465 25747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:00.911653 25747 hybrid_clock.cc:648] HybridClock initialized: now 1786031940911624 us; error 41 us; skew 500 ppm
Aug 06 15:59:00 dist-test-slave-tm1g krb5kdc[25733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031940, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:00.914412 25747 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 15:59:00.915257 25747 webserver.cc:460] Webserver started at http://127.25.31.190:39369/ using document root <none> and password file <none>
I20260806 15:59:00.915827 25747 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:00.915920 25747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:00.916157 25747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:00.918203 25747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "2d4e0b80e01047d08b02a1e53968479a"
format_stamp: "Formatted at 2026-08-06 15:59:00 on dist-test-slave-tm1g"
server_key: "a8a1930142b864c05d2e8ecc35b556cc"
server_key_iv: "50d0314e01e2c0af52b914098d20fa79"
server_key_version: "encryptionkey@0"
I20260806 15:59:00.918831 25747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "2d4e0b80e01047d08b02a1e53968479a"
format_stamp: "Formatted at 2026-08-06 15:59:00 on dist-test-slave-tm1g"
server_key: "a8a1930142b864c05d2e8ecc35b556cc"
server_key_iv: "50d0314e01e2c0af52b914098d20fa79"
server_key_version: "encryptionkey@0"
I20260806 15:59:00.922616 25747 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 15:59:00.924921 25763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:00.926116 25747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:00.926265 25747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "2d4e0b80e01047d08b02a1e53968479a"
format_stamp: "Formatted at 2026-08-06 15:59:00 on dist-test-slave-tm1g"
server_key: "a8a1930142b864c05d2e8ecc35b556cc"
server_key_iv: "50d0314e01e2c0af52b914098d20fa79"
server_key_version: "encryptionkey@0"
I20260806 15:59:00.926451 25747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:00.952224 25747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:00.955425 25747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:00.955639 25747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:00.962791 25747 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:42313
I20260806 15:59:00.962821 25817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:42313 every 8 connection(s)
I20260806 15:59:00.963742 25747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:00.966513 25818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:00.972167 25818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a: Bootstrap starting.
I20260806 15:59:00.974280 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 25747
I20260806 15:59:00.974475 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:00.975003 25818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:00.975899 25818 log.cc:824] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:00.976132 25726 external_mini_cluster.cc:1428] Setting key 828bb92b68924eea7704a4e61f9f7ce6
I20260806 15:59:00.978191 25818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a: No bootstrap required, opened a new log
I20260806 15:59:00.982393 25818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d4e0b80e01047d08b02a1e53968479a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42313 } }
I20260806 15:59:00.982609 25818 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:00.982689 25818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d4e0b80e01047d08b02a1e53968479a, State: Initialized, Role: FOLLOWER
I20260806 15:59:00.983119 25818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d4e0b80e01047d08b02a1e53968479a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42313 } }
I20260806 15:59:00.983235 25818 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:00.983323 25818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:00.983436 25818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:00.984460 25818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d4e0b80e01047d08b02a1e53968479a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42313 } }
I20260806 15:59:00.984807 25818 leader_election.cc:302] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d4e0b80e01047d08b02a1e53968479a; no voters:
I20260806 15:59:00.985193 25818 leader_election.cc:288] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:00.985378 25823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:00.985679 25823 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [term 1 LEADER]: Becoming Leader. State: Replica: 2d4e0b80e01047d08b02a1e53968479a, State: Running, Role: LEADER
I20260806 15:59:00.986035 25823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d4e0b80e01047d08b02a1e53968479a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42313 } }
I20260806 15:59:00.986450 25818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:00.987707 25824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d4e0b80e01047d08b02a1e53968479a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d4e0b80e01047d08b02a1e53968479a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42313 } } }
I20260806 15:59:00.987849 25824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:00.988240 25825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d4e0b80e01047d08b02a1e53968479a. Latest consensus state: current_term: 1 leader_uuid: "2d4e0b80e01047d08b02a1e53968479a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d4e0b80e01047d08b02a1e53968479a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42313 } } }
I20260806 15:59:00.988334 25825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:00.988600 25834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:00.991640 25834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 15:59:00 dist-test-slave-tm1g krb5kdc[25733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:00.997414 25834 catalog_manager.cc:1347] Generated new cluster ID: 4bf2fa28960640eca2225717516009d4
I20260806 15:59:00.997524 25834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:01.003855 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:00.979541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47814 (local address 127.25.31.190:42313)
0806 15:59:00.980024 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:00.980046 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:00.981178 (+ 1132us) server_negotiation.cc:407] Connection header received
0806 15:59:00.984205 (+ 3027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:00.984223 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:00.984613 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:00.984961 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:00.988011 (+ 3050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:00.989593 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:00.991269 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:00.991952 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:00.992136 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:00.997064 (+ 4928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:00.997072 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:00.997087 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:00.997120 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:00.999946 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:01.000601 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:01.000608 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:01.000613 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:01.000761 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:01.001124 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:01.001128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:01.001129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:01.001765 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:01.001983 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:01.002594 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 15:59:01.002893 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":141,"threads_started":1}
I20260806 15:59:01.016193 25834 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:01.017114 25834 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:01.028687 25834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d4e0b80e01047d08b02a1e53968479a: Generated new TSK 0
I20260806 15:59:01.029381 25834 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 15:59:01.063428 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42313
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:01.173038 25845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:01.173264 25845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:01.173343 25845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:01.177337 25845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:01.177390 25845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:01.177481 25845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:01.182542 25845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42313
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:01.183481 25845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:01.184434 25845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:59:01.193995 25845 server_base.cc:1034] running on GCE node
W20260806 15:59:01.193948 25851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:01.193943 25850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:01.194416 25853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:01.194918 25845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:01.195508 25845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:01.196667 25845 hybrid_clock.cc:648] HybridClock initialized: now 1786031941196649 us; error 30 us; skew 500 ppm
Aug 06 15:59:01 dist-test-slave-tm1g krb5kdc[25733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031941, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:01.199465 25845 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 15:59:01.200296 25845 webserver.cc:460] Webserver started at http://127.25.31.129:36229/ using document root <none> and password file <none>
I20260806 15:59:01.200870 25845 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:01.200930 25845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:01.201169 25845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:01.203258 25845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "8217595aa3d64ece9927a739fe84c15c"
format_stamp: "Formatted at 2026-08-06 15:59:01 on dist-test-slave-tm1g"
server_key: "9371a9f91a32a335e2b4d8268349a808"
server_key_iv: "4ae88ae70a2d96cc6978de1d29ec75d4"
server_key_version: "encryptionkey@0"
I20260806 15:59:01.203831 25845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "8217595aa3d64ece9927a739fe84c15c"
format_stamp: "Formatted at 2026-08-06 15:59:01 on dist-test-slave-tm1g"
server_key: "9371a9f91a32a335e2b4d8268349a808"
server_key_iv: "4ae88ae70a2d96cc6978de1d29ec75d4"
server_key_version: "encryptionkey@0"
I20260806 15:59:01.207756 25845 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 15:59:01.210028 25860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:01.211328 25845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:01.211432 25845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "8217595aa3d64ece9927a739fe84c15c"
format_stamp: "Formatted at 2026-08-06 15:59:01 on dist-test-slave-tm1g"
server_key: "9371a9f91a32a335e2b4d8268349a808"
server_key_iv: "4ae88ae70a2d96cc6978de1d29ec75d4"
server_key_version: "encryptionkey@0"
I20260806 15:59:01.211526 25845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:01.247195 25845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:01.249970 25845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:01.250124 25845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:01.250730 25845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:01.251523 25845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:01.251569 25845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:01.251608 25845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:01.251624 25845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:01.260707 25845 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:43755
I20260806 15:59:01.260741 25977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:43755 every 8 connection(s)
I20260806 15:59:01.261582 25845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:01.269682 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 25845
I20260806 15:59:01.269794 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 15:59:01.270064 25726 external_mini_cluster.cc:1428] Setting key b95b83d33018891fc89ef20ca9638222
Aug 06 15:59:01 dist-test-slave-tm1g krb5kdc[25733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031941, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:01.276882 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:01.263360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:38891 (local address 127.25.31.190:42313)
0806 15:59:01.263515 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:01.263519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:01.264289 (+ 770us) server_negotiation.cc:407] Connection header received
0806 15:59:01.265577 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:01.265580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:01.265645 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:01.265752 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:01.266958 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.267610 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.268776 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.269015 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.269131 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:01.271942 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:01.271946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:01.271949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:01.271982 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:01.273776 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:01.274748 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:01.274751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:01.274752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:01.274834 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:01.275361 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:01.275367 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:01.275372 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:01.275817 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:01.275930 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:01.276456 (+ 526us) server_negotiation.cc:299] Negotiation successful
0806 15:59:01.276622 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 15:59:01.277580 25981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:01.263713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42313 (local address 127.25.31.129:38891)
0806 15:59:01.264139 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:01.264181 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:01.265362 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:01.265924 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:01.265930 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:01.266282 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:01.266802 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:01.266815 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.267765 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.267770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:01.268646 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:01.268656 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.269174 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.269178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:01.269350 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:01.269973 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:01.269990 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:01.271767 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:01.273998 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:01.274006 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:01.274181 (+ 175us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:01.274616 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:01.274939 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:01.274943 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:01.274947 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:01.275160 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:01.275951 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:01.275956 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:01.276192 (+ 236us) client_negotiation.cc:769] Sending connection context
0806 15:59:01.276356 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 15:59:01.276639 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":103,"threads_started":1}
I20260806 15:59:01.278946 25979 heartbeater.cc:347] Connected to a master server at 127.25.31.190:42313
I20260806 15:59:01.279217 25979 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:01.279826 25979 heartbeater.cc:511] Master 127.25.31.190:42313 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
I20260806 15:59:01.281606 25781 ts_manager.cc:194] Registered new tserver with Master: 8217595aa3d64ece9927a739fe84c15c (127.25.31.129:43755)
I20260806 15:59:01.282910 25781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:38891
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 15:59:01.301564 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42313
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:01.409797 25984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:01.410039 25984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:01.410081 25984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:01.413910 25984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:01.413957 25984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:01.414017 25984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:01.419135 25984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42313
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:01.420044 25984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:01.420948 25984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:01.431196 25989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:01.431196 25990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:01.431341 25992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:01.431975 25984 server_base.cc:1034] running on GCE node
I20260806 15:59:01.432440 25984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:01.433214 25984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:01.434356 25984 hybrid_clock.cc:648] HybridClock initialized: now 1786031941434333 us; error 35 us; skew 500 ppm
Aug 06 15:59:01 dist-test-slave-tm1g krb5kdc[25733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031941, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:01.437865 25984 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 15:59:01.439131 25984 webserver.cc:460] Webserver started at http://127.25.31.130:37831/ using document root <none> and password file <none>
I20260806 15:59:01.439919 25984 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:01.440028 25984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:01.440315 25984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:01.442900 25984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "9c99123abb0b4bcd831383fab72599ca"
format_stamp: "Formatted at 2026-08-06 15:59:01 on dist-test-slave-tm1g"
server_key: "2ece4fe85f37eae4075c8a64f73b5fc8"
server_key_iv: "4dfa0ff445d1a5731a6823fdce233d19"
server_key_version: "encryptionkey@0"
I20260806 15:59:01.443461 25984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "9c99123abb0b4bcd831383fab72599ca"
format_stamp: "Formatted at 2026-08-06 15:59:01 on dist-test-slave-tm1g"
server_key: "2ece4fe85f37eae4075c8a64f73b5fc8"
server_key_iv: "4dfa0ff445d1a5731a6823fdce233d19"
server_key_version: "encryptionkey@0"
I20260806 15:59:01.447052 25984 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 15:59:01.449218 26001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:01.450443 25984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:01.450556 25984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "9c99123abb0b4bcd831383fab72599ca"
format_stamp: "Formatted at 2026-08-06 15:59:01 on dist-test-slave-tm1g"
server_key: "2ece4fe85f37eae4075c8a64f73b5fc8"
server_key_iv: "4dfa0ff445d1a5731a6823fdce233d19"
server_key_version: "encryptionkey@0"
I20260806 15:59:01.450656 25984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:01.466065 25984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:01.468820 25984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:01.469023 25984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:01.469589 25984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:01.470429 25984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:01.470477 25984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:01.470542 25984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:01.470579 25984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:01.479303 25984 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:44309
I20260806 15:59:01.479348 26120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:44309 every 8 connection(s)
I20260806 15:59:01.480156 25984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:01.487322 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 25984
I20260806 15:59:01.487440 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 15:59:01.487692 25726 external_mini_cluster.cc:1428] Setting key 04e465c2751dc0ce2d76a04edd1175e2
Aug 06 15:59:01 dist-test-slave-tm1g krb5kdc[25733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031941, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:01.494231 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:01.481839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:49897 (local address 127.25.31.190:42313)
0806 15:59:01.481979 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:01.481982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:01.483213 (+ 1231us) server_negotiation.cc:407] Connection header received
0806 15:59:01.484105 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:01.484108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:01.484169 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:01.484274 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:01.485398 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.485940 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.487281 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.487608 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.487720 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:01.490673 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:01.490676 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:01.490679 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:01.490713 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:01.492186 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:01.492715 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:01.492718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:01.492720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:01.492793 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:01.493148 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:01.493151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:01.493153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:01.493332 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:01.493442 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:01.493923 (+ 481us) server_negotiation.cc:299] Negotiation successful
0806 15:59:01.494062 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 15:59:01.495062 26123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:01.482173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42313 (local address 127.25.31.130:49897)
0806 15:59:01.483066 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:01.483109 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:01.483901 (+ 792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:01.484445 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:01.484451 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:01.484794 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:01.485238 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:01.485250 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.486126 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.486129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:01.487164 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:01.487174 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.487775 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.487779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:01.487945 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:01.488620 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:01.488640 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:01.490517 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:01.492311 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:01.492317 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:01.492330 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:01.492596 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:01.492888 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:01.492891 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:01.492893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:01.493044 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:01.493441 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:01.493447 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:01.493686 (+ 239us) client_negotiation.cc:769] Sending connection context
0806 15:59:01.493864 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 15:59:01.494183 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":564,"threads_started":1}
I20260806 15:59:01.496176 26121 heartbeater.cc:347] Connected to a master server at 127.25.31.190:42313
I20260806 15:59:01.496423 26121 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:01.497033 26121 heartbeater.cc:511] Master 127.25.31.190:42313 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
I20260806 15:59:01.498162 25781 ts_manager.cc:194] Registered new tserver with Master: 9c99123abb0b4bcd831383fab72599ca (127.25.31.130:44309)
I20260806 15:59:01.498873 25781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:49897
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 15:59:01.515262 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42313
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:38521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:01.625463 26127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:01.625715 26127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:01.625757 26127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:01.630026 26127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:01.630079 26127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:01.630175 26127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:01.635839 26127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:38521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42313
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:01.636863 26127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:01.637852 26127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:01.647655 26132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:01.647686 26133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:01.647680 26135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:01.648020 26127 server_base.cc:1034] running on GCE node
I20260806 15:59:01.648399 26127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:01.649031 26127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:01.650189 26127 hybrid_clock.cc:648] HybridClock initialized: now 1786031941650153 us; error 49 us; skew 500 ppm
Aug 06 15:59:01 dist-test-slave-tm1g krb5kdc[25733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031941, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:01.652830 26127 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 15:59:01.653730 26127 webserver.cc:460] Webserver started at http://127.25.31.131:44763/ using document root <none> and password file <none>
I20260806 15:59:01.654289 26127 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:01.654398 26127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:01.654626 26127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:01.656600 26127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "f2f1845c068a4b39bb7d504a332603e2"
format_stamp: "Formatted at 2026-08-06 15:59:01 on dist-test-slave-tm1g"
server_key: "df84f7d248b4fd466b3f33aea2007a3b"
server_key_iv: "a620f81c5d8ea87130c83eb48be0a6c5"
server_key_version: "encryptionkey@0"
I20260806 15:59:01.657104 26127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "f2f1845c068a4b39bb7d504a332603e2"
format_stamp: "Formatted at 2026-08-06 15:59:01 on dist-test-slave-tm1g"
server_key: "df84f7d248b4fd466b3f33aea2007a3b"
server_key_iv: "a620f81c5d8ea87130c83eb48be0a6c5"
server_key_version: "encryptionkey@0"
I20260806 15:59:01.660693 26127 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 15:59:01.663008 26143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:01.664146 26127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:01.664278 26127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "f2f1845c068a4b39bb7d504a332603e2"
format_stamp: "Formatted at 2026-08-06 15:59:01 on dist-test-slave-tm1g"
server_key: "df84f7d248b4fd466b3f33aea2007a3b"
server_key_iv: "a620f81c5d8ea87130c83eb48be0a6c5"
server_key_version: "encryptionkey@0"
I20260806 15:59:01.664384 26127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:01.674494 26127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:01.677223 26127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:01.677419 26127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:01.677986 26127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:01.678819 26127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:01.678890 26127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:01.678960 26127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:01.679006 26127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:01.687918 26127 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:33531
I20260806 15:59:01.687961 26264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:33531 every 8 connection(s)
I20260806 15:59:01.688823 26127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 15:59:01.690797 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 26127
I20260806 15:59:01.690917 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 15:59:01.691200 25726 external_mini_cluster.cc:1428] Setting key f5aeddf8629ed76c41151984882a5011
Aug 06 15:59:01 dist-test-slave-tm1g krb5kdc[25733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031941, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:01.702277 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:01.690840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:34009 (local address 127.25.31.190:42313)
0806 15:59:01.691003 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:01.691008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:01.691974 (+ 966us) server_negotiation.cc:407] Connection header received
0806 15:59:01.693546 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:01.693551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:01.693609 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:01.693694 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:01.694853 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.695390 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.696463 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.696680 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.696763 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:01.699127 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:01.699129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:01.699132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:01.699157 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:01.700585 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:01.701022 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:01.701024 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:01.701026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:01.701101 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:01.701420 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:01.701422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:01.701423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:01.701585 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:01.701658 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:01.702069 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 15:59:01.702161 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 15:59:01.703109 26267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:01.691289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42313 (local address 127.25.31.131:34009)
0806 15:59:01.691824 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:01.691881 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:01.693298 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:01.693875 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:01.693884 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:01.694249 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:01.694728 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:01.694741 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.695503 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.695505 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:01.696351 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:01.696359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.696806 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.696809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:01.696941 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:01.697454 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:01.697476 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:01.698982 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:01.700680 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:01.700684 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:01.700694 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:01.700942 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:01.701203 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:01.701205 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:01.701206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:01.701334 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:01.701670 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:01.701674 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:01.701878 (+ 204us) client_negotiation.cc:769] Sending connection context
0806 15:59:01.702022 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 15:59:01.702247 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":148,"threads_started":1}
I20260806 15:59:01.704188 26265 heartbeater.cc:347] Connected to a master server at 127.25.31.190:42313
I20260806 15:59:01.704432 26265 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:01.704895 26265 heartbeater.cc:511] Master 127.25.31.190:42313 requested a full tablet report, sending...
I20260806 15:59:01.705857 25781 ts_manager.cc:194] Registered new tserver with Master: f2f1845c068a4b39bb7d504a332603e2 (127.25.31.131:33531)
I20260806 15:59:01.706437 25781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:34009
I20260806 15:59:01.717064 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 15:59:01 dist-test-slave-tm1g krb5kdc[25733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031941, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 15:59:01 dist-test-slave-tm1g krb5kdc[25733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031941, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.31.129@KRBTEST.COM
I20260806 15:59:01.743309 26272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:01.733789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41428 (local address 127.25.31.129:43755)
0806 15:59:01.734093 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:01.734106 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:01.734130 (+ 24us) server_negotiation.cc:407] Connection header received
0806 15:59:01.734213 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:01.734218 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:01.734484 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:01.734607 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:01.735233 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.735998 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.736782 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.737139 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.737222 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:01.739588 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:01.739602 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:01.739618 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:01.739652 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:01.741357 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:01.741802 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:01.741815 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:01.741816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:01.741896 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:01.742172 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:01.742175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:01.742177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:01.742495 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:01.742711 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:01.742882 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 15:59:01.743042 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":100,"threads_started":1}
W20260806 15:59:01.744580 25889 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41428
Aug 06 15:59:01 dist-test-slave-tm1g krb5kdc[25733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 15:59:01 dist-test-slave-tm1g krb5kdc[25733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.129@KRBTEST.COM
I20260806 15:59:01.771829 26272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:01.763265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41434 (local address 127.25.31.129:43755)
0806 15:59:01.763434 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:01.763439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:01.763565 (+ 126us) server_negotiation.cc:407] Connection header received
0806 15:59:01.763752 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:01.763755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:01.763799 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:01.763884 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:01.764605 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.765148 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.766008 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:01.766257 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:01.766432 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:01.768874 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:01.768880 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:01.768887 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:01.768912 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:01.770147 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:01.770605 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:01.770609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:01.770613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:01.770693 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:01.770977 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:01.770981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:01.770983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:01.771152 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:01.771293 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:01.771534 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 15:59:01.771659 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 15:59:01.772866 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 25845
I20260806 15:59:01.779646 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 25984
I20260806 15:59:01.787125 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 26127
I20260806 15:59:01.792717 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 25747
2026-08-06T15:59:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2202 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:59:01 dist-test-slave-tm1g krb5kdc[26279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:59:01 dist-test-slave-tm1g krb5kdc[26279](info): set up 2 sockets
Aug 06 15:59:01 dist-test-slave-tm1g krb5kdc[26279](info): commencing operation
krb5kdc: starting...
W20260806 15:59:02.842701 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 15:59:02 dist-test-slave-tm1g krb5kdc[26279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:59:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 15:59:02.961287 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:46803
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:46803
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:03.071026 26292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:03.071271 26292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:03.071311 26292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:03.075207 26292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:03.075254 26292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:03.075273 26292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:03.075289 26292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:03.075304 26292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:03.080420 26292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:46803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:46803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:03.081347 26292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:03.082274 26292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:03.090507 26298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:03.090507 26297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:03.090584 26300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:03.090728 26292 server_base.cc:1034] running on GCE node
I20260806 15:59:03.091333 26292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:03.092435 26292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:03.093631 26292 hybrid_clock.cc:648] HybridClock initialized: now 1786031943093614 us; error 31 us; skew 500 ppm
Aug 06 15:59:03 dist-test-slave-tm1g krb5kdc[26279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031943, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:03.101076 26292 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 15:59:03.102073 26292 webserver.cc:460] Webserver started at http://127.25.31.190:34183/ using document root <none> and password file <none>
I20260806 15:59:03.102672 26292 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:03.102733 26292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:03.102900 26292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:03.104863 26292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "6949aacbec524333826505974c293efc"
format_stamp: "Formatted at 2026-08-06 15:59:03 on dist-test-slave-tm1g"
server_key: "db6cdbde99bf07e8b0de0c1d2a739075"
server_key_iv: "24c6d8df6b6536214e0ab806f062d187"
server_key_version: "encryptionkey@0"
I20260806 15:59:03.105379 26292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "6949aacbec524333826505974c293efc"
format_stamp: "Formatted at 2026-08-06 15:59:03 on dist-test-slave-tm1g"
server_key: "db6cdbde99bf07e8b0de0c1d2a739075"
server_key_iv: "24c6d8df6b6536214e0ab806f062d187"
server_key_version: "encryptionkey@0"
I20260806 15:59:03.108979 26292 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260806 15:59:03.111253 26308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:03.112407 26292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:03.112506 26292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "6949aacbec524333826505974c293efc"
format_stamp: "Formatted at 2026-08-06 15:59:03 on dist-test-slave-tm1g"
server_key: "db6cdbde99bf07e8b0de0c1d2a739075"
server_key_iv: "24c6d8df6b6536214e0ab806f062d187"
server_key_version: "encryptionkey@0"
I20260806 15:59:03.112596 26292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:03.134133 26292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:03.137053 26292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:03.137199 26292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:03.144155 26292 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:46803
I20260806 15:59:03.144181 26363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:46803 every 8 connection(s)
I20260806 15:59:03.145083 26292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:03.147130 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 26292
I20260806 15:59:03.147215 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:03.147473 25726 external_mini_cluster.cc:1428] Setting key f146f1f4b3952dc29af426370059ba5f
I20260806 15:59:03.147944 26364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:03.154914 26364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc: Bootstrap starting.
Aug 06 15:59:03 dist-test-slave-tm1g krb5kdc[26279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:03.157232 26364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:03.158170 26364 log.cc:824] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:03.160347 26364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc: No bootstrap required, opened a new log
I20260806 15:59:03.162285 26367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:03.148838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43664 (local address 127.25.31.190:46803)
0806 15:59:03.149265 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:03.149287 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:03.149331 (+ 44us) server_negotiation.cc:407] Connection header received
0806 15:59:03.150121 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:03.150144 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:03.150575 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:03.150947 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:03.151558 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.153059 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.153902 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.154615 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.154775 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:03.157112 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:03.157122 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:03.157141 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:03.157176 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:03.159066 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:03.159466 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:03.159471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:03.159477 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:03.159618 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:03.159882 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:03.159885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:03.159886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:03.160361 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:03.160541 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:03.160828 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 15:59:03.161124 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":120,"threads_started":1}
I20260806 15:59:03.164106 26364 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6949aacbec524333826505974c293efc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 46803 } }
I20260806 15:59:03.164397 26364 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:03.164458 26364 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6949aacbec524333826505974c293efc, State: Initialized, Role: FOLLOWER
I20260806 15:59:03.164855 26364 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6949aacbec524333826505974c293efc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 46803 } }
I20260806 15:59:03.164970 26364 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:03.165052 26364 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:03.165162 26364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:03.166083 26364 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6949aacbec524333826505974c293efc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 46803 } }
I20260806 15:59:03.166483 26364 leader_election.cc:302] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6949aacbec524333826505974c293efc; no voters:
I20260806 15:59:03.166826 26364 leader_election.cc:288] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:03.166949 26370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:03.167174 26370 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [term 1 LEADER]: Becoming Leader. State: Replica: 6949aacbec524333826505974c293efc, State: Running, Role: LEADER
I20260806 15:59:03.167457 26370 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6949aacbec524333826505974c293efc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 46803 } }
I20260806 15:59:03.167996 26364 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:03.168493 26370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6949aacbec524333826505974c293efc. Latest consensus state: current_term: 1 leader_uuid: "6949aacbec524333826505974c293efc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6949aacbec524333826505974c293efc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 46803 } } }
I20260806 15:59:03.168630 26370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:03.169059 26373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6949aacbec524333826505974c293efc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6949aacbec524333826505974c293efc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 46803 } } }
I20260806 15:59:03.169219 26373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:03.169411 26376 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:03.172072 26376 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:03.177724 26376 catalog_manager.cc:1347] Generated new cluster ID: 67b4983f13624ba5a3dbeb40c1bf7528
I20260806 15:59:03.177795 26376 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:03.186324 26376 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:03.187448 26376 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:03.194288 26376 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6949aacbec524333826505974c293efc: Generated new TSK 0
I20260806 15:59:03.195042 26376 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 15:59:03.227458 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:46803
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:03.337082 26392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:03.337313 26392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:03.337357 26392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:03.341297 26392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:03.341349 26392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:03.341405 26392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:03.346544 26392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:46803
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:03.347440 26392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:03.348366 26392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:03.357888 26397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:03.357896 26401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:03.357901 26398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:03.357932 26392 server_base.cc:1034] running on GCE node
I20260806 15:59:03.358563 26392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:03.359117 26392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:03.360318 26392 hybrid_clock.cc:648] HybridClock initialized: now 1786031943360288 us; error 42 us; skew 500 ppm
Aug 06 15:59:03 dist-test-slave-tm1g krb5kdc[26279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031943, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:03.363128 26392 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 15:59:03.363948 26392 webserver.cc:460] Webserver started at http://127.25.31.129:36767/ using document root <none> and password file <none>
I20260806 15:59:03.364490 26392 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:03.364552 26392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:03.364784 26392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:03.366806 26392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "dd1be0e0c4d2471bb2ae0b576f125e4b"
format_stamp: "Formatted at 2026-08-06 15:59:03 on dist-test-slave-tm1g"
server_key: "1ab1f4de85da74a74b0b9f3c48f7ef25"
server_key_iv: "b8b8079c2e34aa65011609be52edc206"
server_key_version: "encryptionkey@0"
I20260806 15:59:03.367376 26392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "dd1be0e0c4d2471bb2ae0b576f125e4b"
format_stamp: "Formatted at 2026-08-06 15:59:03 on dist-test-slave-tm1g"
server_key: "1ab1f4de85da74a74b0b9f3c48f7ef25"
server_key_iv: "b8b8079c2e34aa65011609be52edc206"
server_key_version: "encryptionkey@0"
I20260806 15:59:03.371269 26392 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 15:59:03.373592 26409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:03.374822 26392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:03.374953 26392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "dd1be0e0c4d2471bb2ae0b576f125e4b"
format_stamp: "Formatted at 2026-08-06 15:59:03 on dist-test-slave-tm1g"
server_key: "1ab1f4de85da74a74b0b9f3c48f7ef25"
server_key_iv: "b8b8079c2e34aa65011609be52edc206"
server_key_version: "encryptionkey@0"
I20260806 15:59:03.375051 26392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:03.395282 26392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:03.398155 26392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:03.398375 26392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:03.398960 26392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:03.399750 26392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:03.399797 26392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:03.399873 26392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:03.399906 26392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:03.408682 26392 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:34717
I20260806 15:59:03.408715 26526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:34717 every 8 connection(s)
I20260806 15:59:03.409614 26392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:03.413580 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 26392
I20260806 15:59:03.413697 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 15:59:03.413986 25726 external_mini_cluster.cc:1428] Setting key 309bdef4aff05e8d6121b51662ddc50f
Aug 06 15:59:03 dist-test-slave-tm1g krb5kdc[26279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031943, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:03.423804 26367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:03.411454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:38001 (local address 127.25.31.190:46803)
0806 15:59:03.411616 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:03.411620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:03.412368 (+ 748us) server_negotiation.cc:407] Connection header received
0806 15:59:03.413194 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:03.413197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:03.413243 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:03.413316 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:03.414768 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.415311 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.416515 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.416842 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.416969 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:03.419917 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:03.419923 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:03.419926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:03.419954 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:03.421669 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:03.422147 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:03.422151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:03.422153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:03.422247 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:03.422616 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:03.422621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:03.422623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:03.422867 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:03.422977 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:03.423458 (+ 481us) server_negotiation.cc:299] Negotiation successful
0806 15:59:03.423598 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
I20260806 15:59:03.424438 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:03.411778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:46803 (local address 127.25.31.129:38001)
0806 15:59:03.412206 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:03.412248 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:03.413007 (+ 759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:03.413530 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:03.413539 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:03.414028 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:03.414616 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:03.414630 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.415453 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.415461 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:03.416383 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:03.416392 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.417182 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.417189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:03.417425 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:03.418015 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:03.418042 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:03.419748 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:03.421785 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:03.421789 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:03.421799 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:03.422041 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:03.422381 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:03.422383 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:03.422384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:03.422517 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:03.422974 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:03.422978 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:03.423213 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 15:59:03.423356 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 15:59:03.423595 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":110,"threads_started":1}
I20260806 15:59:03.425691 26527 heartbeater.cc:347] Connected to a master server at 127.25.31.190:46803
I20260806 15:59:03.425930 26527 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:03.426455 26527 heartbeater.cc:511] Master 127.25.31.190:46803 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
I20260806 15:59:03.428105 26326 ts_manager.cc:194] Registered new tserver with Master: dd1be0e0c4d2471bb2ae0b576f125e4b (127.25.31.129:34717)
I20260806 15:59:03.429340 26326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:38001
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 15:59:03.441764 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:46803
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:03.549568 26532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:03.549854 26532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:03.549939 26532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:03.553979 26532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:03.554064 26532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:03.554158 26532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:03.559347 26532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:46803
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:03.560352 26532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:03.561334 26532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:03.570779 26538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:03.570824 26540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:03.570788 26537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:03.571123 26532 server_base.cc:1034] running on GCE node
I20260806 15:59:03.571527 26532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:03.572129 26532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:03.573334 26532 hybrid_clock.cc:648] HybridClock initialized: now 1786031943573316 us; error 28 us; skew 500 ppm
Aug 06 15:59:03 dist-test-slave-tm1g krb5kdc[26279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031943, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:03.576078 26532 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 15:59:03.576897 26532 webserver.cc:460] Webserver started at http://127.25.31.130:38021/ using document root <none> and password file <none>
I20260806 15:59:03.577499 26532 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:03.577632 26532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:03.577927 26532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:03.580082 26532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "6c5c0b97905c4e7eab268e0ba7365259"
format_stamp: "Formatted at 2026-08-06 15:59:03 on dist-test-slave-tm1g"
server_key: "56035e80cb1f24e61384c7bc62e0e00b"
server_key_iv: "a011ac9b588a6cb77129e7b7cff737ed"
server_key_version: "encryptionkey@0"
I20260806 15:59:03.580705 26532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "6c5c0b97905c4e7eab268e0ba7365259"
format_stamp: "Formatted at 2026-08-06 15:59:03 on dist-test-slave-tm1g"
server_key: "56035e80cb1f24e61384c7bc62e0e00b"
server_key_iv: "a011ac9b588a6cb77129e7b7cff737ed"
server_key_version: "encryptionkey@0"
I20260806 15:59:03.584247 26532 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 15:59:03.586594 26548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:03.587829 26532 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 15:59:03.587945 26532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "6c5c0b97905c4e7eab268e0ba7365259"
format_stamp: "Formatted at 2026-08-06 15:59:03 on dist-test-slave-tm1g"
server_key: "56035e80cb1f24e61384c7bc62e0e00b"
server_key_iv: "a011ac9b588a6cb77129e7b7cff737ed"
server_key_version: "encryptionkey@0"
I20260806 15:59:03.588056 26532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:03.605890 26532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:03.608868 26532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:03.609072 26532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:03.609645 26532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:03.610502 26532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:03.610551 26532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:03.610641 26532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:03.610677 26532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:03.619879 26532 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:46609
I20260806 15:59:03.619910 26664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:46609 every 8 connection(s)
I20260806 15:59:03.620729 26532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:03.627292 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 26532
I20260806 15:59:03.627421 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 15:59:03.627694 25726 external_mini_cluster.cc:1428] Setting key 7c2974aae1350ecc39aeed9648caca21
Aug 06 15:59:03 dist-test-slave-tm1g krb5kdc[26279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031943, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:03.634169 26367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:03.622446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:37865 (local address 127.25.31.190:46803)
0806 15:59:03.622586 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:03.622590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:03.623406 (+ 816us) server_negotiation.cc:407] Connection header received
0806 15:59:03.624308 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:03.624311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:03.624371 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:03.624471 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:03.625594 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.626133 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.627363 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.627680 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.627791 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:03.630466 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:03.630472 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:03.630478 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:03.630512 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:03.632028 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:03.632622 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:03.632624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:03.632625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:03.632699 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:03.633081 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:03.633084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:03.633085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:03.633245 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:03.633316 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:03.633958 (+ 642us) server_negotiation.cc:299] Negotiation successful
0806 15:59:03.634055 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 15:59:03.635376 26667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:03.622826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:46803 (local address 127.25.31.130:37865)
0806 15:59:03.623264 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:03.623305 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:03.624101 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:03.624653 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:03.624659 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:03.624991 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:03.625447 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:03.625467 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.626256 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.626260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:03.627229 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:03.627238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.627828 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.627831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:03.628005 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:03.628650 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:03.628669 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:03.630237 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:03.632141 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:03.632149 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:03.632165 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:03.632517 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:03.632784 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:03.632790 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:03.632795 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:03.632985 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:03.633350 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:03.633359 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:03.633682 (+ 323us) client_negotiation.cc:769] Sending connection context
0806 15:59:03.633893 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 15:59:03.634247 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":110,"threads_started":1}
I20260806 15:59:03.636484 26665 heartbeater.cc:347] Connected to a master server at 127.25.31.190:46803
I20260806 15:59:03.636726 26665 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:03.637187 26665 heartbeater.cc:511] Master 127.25.31.190:46803 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
I20260806 15:59:03.638247 26326 ts_manager.cc:194] Registered new tserver with Master: 6c5c0b97905c4e7eab268e0ba7365259 (127.25.31.130:46609)
I20260806 15:59:03.638855 26326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:37865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 15:59:03.655114 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:46803
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:40109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:03.767252 26670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:03.767552 26670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:03.767638 26670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:03.771852 26670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:03.771925 26670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:03.772014 26670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:03.777076 26670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:40109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:46803
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:03.778057 26670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:03.779145 26670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:03.788501 26676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:03.788515 26678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:03.788512 26675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:03.788717 26670 server_base.cc:1034] running on GCE node
I20260806 15:59:03.789215 26670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:03.789753 26670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:03.790940 26670 hybrid_clock.cc:648] HybridClock initialized: now 1786031943790917 us; error 32 us; skew 500 ppm
Aug 06 15:59:03 dist-test-slave-tm1g krb5kdc[26279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031943, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:03.793630 26670 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 15:59:03.794548 26670 webserver.cc:460] Webserver started at http://127.25.31.131:37783/ using document root <none> and password file <none>
I20260806 15:59:03.795094 26670 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:03.795186 26670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:03.795420 26670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:03.797361 26670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "857019d5412a4a529869fe7bc376be5f"
format_stamp: "Formatted at 2026-08-06 15:59:03 on dist-test-slave-tm1g"
server_key: "9cad85e475b609f9495fb52902379b50"
server_key_iv: "863d5c47a10cd282c5bf43b8f7b75634"
server_key_version: "encryptionkey@0"
I20260806 15:59:03.797930 26670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "857019d5412a4a529869fe7bc376be5f"
format_stamp: "Formatted at 2026-08-06 15:59:03 on dist-test-slave-tm1g"
server_key: "9cad85e475b609f9495fb52902379b50"
server_key_iv: "863d5c47a10cd282c5bf43b8f7b75634"
server_key_version: "encryptionkey@0"
I20260806 15:59:03.801564 26670 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 15:59:03.804029 26685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:03.805158 26670 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 15:59:03.805291 26670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "857019d5412a4a529869fe7bc376be5f"
format_stamp: "Formatted at 2026-08-06 15:59:03 on dist-test-slave-tm1g"
server_key: "9cad85e475b609f9495fb52902379b50"
server_key_iv: "863d5c47a10cd282c5bf43b8f7b75634"
server_key_version: "encryptionkey@0"
I20260806 15:59:03.805413 26670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:03.823251 26670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:03.826382 26670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:03.826581 26670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:03.827139 26670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:03.827914 26670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:03.828001 26670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:03.828073 26670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:03.828117 26670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:03.836670 26670 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:40257
I20260806 15:59:03.836691 26801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:40257 every 8 connection(s)
I20260806 15:59:03.837524 26670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 15:59:03.840857 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 26670
I20260806 15:59:03.840972 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 15:59:03.841259 25726 external_mini_cluster.cc:1428] Setting key b687afce5f9c23d363759f03281db17a
Aug 06 15:59:03 dist-test-slave-tm1g krb5kdc[26279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031943, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:03.850567 26367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:03.839311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:41403 (local address 127.25.31.190:46803)
0806 15:59:03.839453 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:03.839457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:03.840459 (+ 1002us) server_negotiation.cc:407] Connection header received
0806 15:59:03.841597 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:03.841601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:03.841662 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:03.841748 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:03.843212 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.843792 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.844881 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.845099 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.845179 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:03.847484 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:03.847486 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:03.847489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:03.847510 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:03.848727 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:03.849163 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:03.849165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:03.849167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:03.849237 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:03.849562 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:03.849566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:03.849567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:03.849771 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:03.849862 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:03.850267 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 15:59:03.850443 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 15:59:03.851293 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:03.839719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:46803 (local address 127.25.31.131:41403)
0806 15:59:03.840265 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:03.840332 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:03.841390 (+ 1058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:03.841942 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:03.841949 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:03.842504 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:03.843038 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:03.843050 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.843902 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.843905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:03.844768 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:03.844776 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.845223 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.845226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:03.845366 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:03.845913 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:03.845928 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:03.847335 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:03.848825 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:03.848829 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:03.848839 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:03.849073 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:03.849314 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:03.849316 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:03.849317 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:03.849438 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:03.849868 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:03.849872 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:03.850072 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 15:59:03.850215 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 15:59:03.850509 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":148,"threads_started":1}
I20260806 15:59:03.852339 26802 heartbeater.cc:347] Connected to a master server at 127.25.31.190:46803
I20260806 15:59:03.852562 26802 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:03.853010 26802 heartbeater.cc:511] Master 127.25.31.190:46803 requested a full tablet report, sending...
I20260806 15:59:03.853983 26326 ts_manager.cc:194] Registered new tserver with Master: 857019d5412a4a529869fe7bc376be5f (127.25.31.131:40257)
I20260806 15:59:03.854671 26326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:41403
I20260806 15:59:03.855057 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:59:03.865204 26367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:03.858647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43670 (local address 127.25.31.190:46803)
0806 15:59:03.858757 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:03.858761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:03.858970 (+ 209us) server_negotiation.cc:407] Connection header received
0806 15:59:03.859058 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:03.859060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:03.859093 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:03.859157 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:03.859790 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.860256 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.861004 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.861200 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.861271 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:03.862571 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:03.862574 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:03.862576 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:03.862595 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:03.863848 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:03.864246 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:03.864248 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:03.864249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:03.864313 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:03.864544 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:03.864546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:03.864547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:03.864724 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:03.864820 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:03.865021 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 15:59:03.865102 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 15:59:03.869845 26326 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:59:03.872933 26326 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:03.886929 26817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:03.881032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:34717 (local address 127.0.0.1:49958)
0806 15:59:03.881811 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:03.881826 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:03.881913 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:03.882983 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:03.882986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:03.883005 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:03.883100 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:03.883107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.884211 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.884215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:03.885265 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:03.885274 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.886353 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.886357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:03.886525 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:03.886543 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 15:59:03.886609 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 15:59:03.886767 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":110,"thread_start_us":82,"threads_started":1}
I20260806 15:59:03.886971 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:03.882016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49958 (local address 127.25.31.129:34717)
0806 15:59:03.882284 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:03.882299 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:03.882591 (+ 292us) server_negotiation.cc:407] Connection header received
0806 15:59:03.882670 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:03.882676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:03.882859 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:03.883004 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:03.883183 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.884070 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.885425 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.886182 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.886507 (+ 325us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:03.886659 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:03.886713 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 15:59:03.886815 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":100,"threads_started":1}
I20260806 15:59:03.887104 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:03.881046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38972 (local address 127.25.31.131:40257)
0806 15:59:03.881784 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:03.881794 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:03.881818 (+ 24us) server_negotiation.cc:407] Connection header received
0806 15:59:03.881897 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:03.881902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:03.882098 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:03.882634 (+ 536us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:03.882798 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.883816 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.885412 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.886136 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.886250 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:03.886412 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:03.886844 (+ 432us) server_negotiation.cc:299] Negotiation successful
0806 15:59:03.886934 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":133,"threads_started":1}
I20260806 15:59:03.887423 26815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:03.881601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:46609 (local address 127.0.0.1:45016)
0806 15:59:03.882522 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:03.882536 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:03.882625 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:03.883250 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:03.883253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:03.883271 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:03.883377 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:03.883384 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.884631 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.884634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:03.885739 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:03.885748 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.886990 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.886997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:03.887119 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:03.887148 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 15:59:03.887221 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 15:59:03.887288 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":859,"thread_start_us":87,"threads_started":1}
I20260806 15:59:03.887425 26814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:03.880909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:40257 (local address 127.0.0.1:38972)
0806 15:59:03.881359 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:03.881407 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:03.881507 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:03.882279 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:03.882284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:03.882307 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:03.882464 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:03.882471 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.883957 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.883960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:03.885262 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:03.885273 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.886259 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.886262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:03.886404 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:03.886508 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 15:59:03.886612 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 15:59:03.886762 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":76,"thread_start_us":82,"threads_started":1}
I20260806 15:59:03.887853 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:03.881768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45016 (local address 127.25.31.130:46609)
0806 15:59:03.882343 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:03.882355 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:03.882712 (+ 357us) server_negotiation.cc:407] Connection header received
0806 15:59:03.882786 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:03.882790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:03.882961 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:03.883085 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:03.883503 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.884493 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.885863 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:03.886707 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:03.886807 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:03.886924 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:03.887607 (+ 683us) server_negotiation.cc:299] Negotiation successful
0806 15:59:03.887711 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":102,"threads_started":1}
I20260806 15:59:03.890149 26457 tablet_service.cc:1467] Processing CreateTablet for tablet e0969d36de2643a19773325ae32cc208 (DEFAULT_TABLE table=test-table [id=4946410fcbb74a6c80490a2876bdbfdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:03.890179 26734 tablet_service.cc:1467] Processing CreateTablet for tablet e0969d36de2643a19773325ae32cc208 (DEFAULT_TABLE table=test-table [id=4946410fcbb74a6c80490a2876bdbfdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:03.890179 26597 tablet_service.cc:1467] Processing CreateTablet for tablet e0969d36de2643a19773325ae32cc208 (DEFAULT_TABLE table=test-table [id=4946410fcbb74a6c80490a2876bdbfdb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:03.891211 26457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0969d36de2643a19773325ae32cc208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:03.891219 26597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0969d36de2643a19773325ae32cc208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:03.891227 26734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0969d36de2643a19773325ae32cc208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:03.897583 26820 tablet_bootstrap.cc:492] T e0969d36de2643a19773325ae32cc208 P dd1be0e0c4d2471bb2ae0b576f125e4b: Bootstrap starting.
I20260806 15:59:03.898594 26821 tablet_bootstrap.cc:492] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259: Bootstrap starting.
I20260806 15:59:03.899714 26820 tablet_bootstrap.cc:654] T e0969d36de2643a19773325ae32cc208 P dd1be0e0c4d2471bb2ae0b576f125e4b: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:03.900484 26821 tablet_bootstrap.cc:654] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:03.900779 26820 log.cc:824] T e0969d36de2643a19773325ae32cc208 P dd1be0e0c4d2471bb2ae0b576f125e4b: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:03.901185 26821 log.cc:824] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:03.901841 26822 tablet_bootstrap.cc:492] T e0969d36de2643a19773325ae32cc208 P 857019d5412a4a529869fe7bc376be5f: Bootstrap starting.
I20260806 15:59:03.902757 26820 tablet_bootstrap.cc:492] T e0969d36de2643a19773325ae32cc208 P dd1be0e0c4d2471bb2ae0b576f125e4b: No bootstrap required, opened a new log
I20260806 15:59:03.903134 26820 ts_tablet_manager.cc:1397] T e0969d36de2643a19773325ae32cc208 P dd1be0e0c4d2471bb2ae0b576f125e4b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 15:59:03.903237 26821 tablet_bootstrap.cc:492] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259: No bootstrap required, opened a new log
I20260806 15:59:03.903421 26821 ts_tablet_manager.cc:1397] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260806 15:59:03.903839 26822 tablet_bootstrap.cc:654] T e0969d36de2643a19773325ae32cc208 P 857019d5412a4a529869fe7bc376be5f: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:03.904759 26822 log.cc:824] T e0969d36de2643a19773325ae32cc208 P 857019d5412a4a529869fe7bc376be5f: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:03.906345 26822 tablet_bootstrap.cc:492] T e0969d36de2643a19773325ae32cc208 P 857019d5412a4a529869fe7bc376be5f: No bootstrap required, opened a new log
I20260806 15:59:03.906543 26822 ts_tablet_manager.cc:1397] T e0969d36de2643a19773325ae32cc208 P 857019d5412a4a529869fe7bc376be5f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 15:59:03.907110 26820 raft_consensus.cc:348] T e0969d36de2643a19773325ae32cc208 P dd1be0e0c4d2471bb2ae0b576f125e4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857019d5412a4a529869fe7bc376be5f" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40257 } } peers { permanent_uuid: "dd1be0e0c4d2471bb2ae0b576f125e4b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34717 } } peers { permanent_uuid: "6c5c0b97905c4e7eab268e0ba7365259" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 46609 } }
I20260806 15:59:03.907110 26821 raft_consensus.cc:348] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857019d5412a4a529869fe7bc376be5f" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40257 } } peers { permanent_uuid: "dd1be0e0c4d2471bb2ae0b576f125e4b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34717 } } peers { permanent_uuid: "6c5c0b97905c4e7eab268e0ba7365259" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 46609 } }
I20260806 15:59:03.907379 26820 raft_consensus.cc:374] T e0969d36de2643a19773325ae32cc208 P dd1be0e0c4d2471bb2ae0b576f125e4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:03.907389 26821 raft_consensus.cc:374] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:03.907442 26820 raft_consensus.cc:729] T e0969d36de2643a19773325ae32cc208 P dd1be0e0c4d2471bb2ae0b576f125e4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd1be0e0c4d2471bb2ae0b576f125e4b, State: Initialized, Role: FOLLOWER
I20260806 15:59:03.907442 26821 raft_consensus.cc:729] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c5c0b97905c4e7eab268e0ba7365259, State: Initialized, Role: FOLLOWER
I20260806 15:59:03.907893 26820 consensus_queue.cc:260] T e0969d36de2643a19773325ae32cc208 P dd1be0e0c4d2471bb2ae0b576f125e4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857019d5412a4a529869fe7bc376be5f" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40257 } } peers { permanent_uuid: "dd1be0e0c4d2471bb2ae0b576f125e4b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34717 } } peers { permanent_uuid: "6c5c0b97905c4e7eab268e0ba7365259" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 46609 } }
I20260806 15:59:03.907884 26821 consensus_queue.cc:260] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857019d5412a4a529869fe7bc376be5f" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40257 } } peers { permanent_uuid: "dd1be0e0c4d2471bb2ae0b576f125e4b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34717 } } peers { permanent_uuid: "6c5c0b97905c4e7eab268e0ba7365259" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 46609 } }
I20260806 15:59:03.908849 26527 heartbeater.cc:503] Master 127.25.31.190:46803 was elected leader, sending a full tablet report...
I20260806 15:59:03.908875 26665 heartbeater.cc:503] Master 127.25.31.190:46803 was elected leader, sending a full tablet report...
I20260806 15:59:03.908946 26820 ts_tablet_manager.cc:1428] T e0969d36de2643a19773325ae32cc208 P dd1be0e0c4d2471bb2ae0b576f125e4b: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260806 15:59:03.908946 26821 ts_tablet_manager.cc:1428] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260806 15:59:03.910744 26528 tablet.cc:2369] T e0969d36de2643a19773325ae32cc208 P dd1be0e0c4d2471bb2ae0b576f125e4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:59:03.910933 26822 raft_consensus.cc:348] T e0969d36de2643a19773325ae32cc208 P 857019d5412a4a529869fe7bc376be5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857019d5412a4a529869fe7bc376be5f" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40257 } } peers { permanent_uuid: "dd1be0e0c4d2471bb2ae0b576f125e4b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34717 } } peers { permanent_uuid: "6c5c0b97905c4e7eab268e0ba7365259" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 46609 } }
I20260806 15:59:03.911185 26822 raft_consensus.cc:374] T e0969d36de2643a19773325ae32cc208 P 857019d5412a4a529869fe7bc376be5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:03.911265 26822 raft_consensus.cc:729] T e0969d36de2643a19773325ae32cc208 P 857019d5412a4a529869fe7bc376be5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857019d5412a4a529869fe7bc376be5f, State: Initialized, Role: FOLLOWER
I20260806 15:59:03.911650 26822 consensus_queue.cc:260] T e0969d36de2643a19773325ae32cc208 P 857019d5412a4a529869fe7bc376be5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857019d5412a4a529869fe7bc376be5f" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40257 } } peers { permanent_uuid: "dd1be0e0c4d2471bb2ae0b576f125e4b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34717 } } peers { permanent_uuid: "6c5c0b97905c4e7eab268e0ba7365259" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 46609 } }
I20260806 15:59:03.912425 26802 heartbeater.cc:503] Master 127.25.31.190:46803 was elected leader, sending a full tablet report...
I20260806 15:59:03.913399 26822 ts_tablet_manager.cc:1428] T e0969d36de2643a19773325ae32cc208 P 857019d5412a4a529869fe7bc376be5f: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260806 15:59:04.060643 26826 raft_consensus.cc:482] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:04.060917 26826 raft_consensus.cc:504] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857019d5412a4a529869fe7bc376be5f" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40257 } } peers { permanent_uuid: "dd1be0e0c4d2471bb2ae0b576f125e4b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34717 } } peers { permanent_uuid: "6c5c0b97905c4e7eab268e0ba7365259" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 46609 } }
I20260806 15:59:04.062119 26826 leader_election.cc:288] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 857019d5412a4a529869fe7bc376be5f (127.25.31.131:40257), dd1be0e0c4d2471bb2ae0b576f125e4b (127.25.31.129:34717)
I20260806 15:59:04.065946 26667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:04.062467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:34717 (local address 127.25.31.130:39239)
0806 15:59:04.062699 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:04.062716 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:04.062857 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:04.063141 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:04.063144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:04.063162 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:04.063288 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:04.063294 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:04.064042 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:04.064045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:04.064856 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:04.064865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:04.065511 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:04.065515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:04.065621 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:04.065649 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 15:59:04.065732 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 15:59:04.065807 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":20}
I20260806 15:59:04.065946 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:04.062553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:39239 (local address 127.25.31.129:34717)
0806 15:59:04.062732 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:04.062737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:04.062817 (+ 80us) server_negotiation.cc:407] Connection header received
0806 15:59:04.062983 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:04.062986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:04.063035 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:04.063127 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:04.063403 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:04.063920 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:04.064980 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:04.065389 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:04.065523 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:04.065580 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:04.065760 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 15:59:04.065825 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 15:59:04.066155 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:04.062554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:47247 (local address 127.25.31.131:40257)
0806 15:59:04.062698 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:04.062703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:04.063023 (+ 320us) server_negotiation.cc:407] Connection header received
0806 15:59:04.063120 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:04.063123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:04.063173 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:04.063258 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:04.063548 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:04.064037 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:04.065079 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:04.065516 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:04.065641 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:04.065694 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:04.065913 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 15:59:04.066009 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 15:59:04.066510 26829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:04.062467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:40257 (local address 127.25.31.130:47247)
0806 15:59:04.062907 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:04.062923 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:04.063027 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:04.063268 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:04.063271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:04.063286 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:04.063427 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:04.063432 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:04.064145 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:04.064150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:04.064947 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:04.064955 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:04.065641 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:04.065645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:04.065772 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:04.065795 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 15:59:04.066288 (+ 493us) client_negotiation.cc:240] Negotiation successful
0806 15:59:04.066381 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":74,"thread_start_us":156,"threads_started":1}
I20260806 15:59:04.066689 26479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0969d36de2643a19773325ae32cc208" candidate_uuid: "6c5c0b97905c4e7eab268e0ba7365259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd1be0e0c4d2471bb2ae0b576f125e4b" is_pre_election: true
I20260806 15:59:04.067003 26479 raft_consensus.cc:2454] T e0969d36de2643a19773325ae32cc208 P dd1be0e0c4d2471bb2ae0b576f125e4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c5c0b97905c4e7eab268e0ba7365259 in term 0.
I20260806 15:59:04.067206 26755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0969d36de2643a19773325ae32cc208" candidate_uuid: "6c5c0b97905c4e7eab268e0ba7365259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857019d5412a4a529869fe7bc376be5f" is_pre_election: true
I20260806 15:59:04.067495 26755 raft_consensus.cc:2454] T e0969d36de2643a19773325ae32cc208 P 857019d5412a4a529869fe7bc376be5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c5c0b97905c4e7eab268e0ba7365259 in term 0.
I20260806 15:59:04.067490 26552 leader_election.cc:302] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c5c0b97905c4e7eab268e0ba7365259, dd1be0e0c4d2471bb2ae0b576f125e4b; no voters:
I20260806 15:59:04.067763 26826 raft_consensus.cc:2781] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:04.067871 26826 raft_consensus.cc:482] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:04.067947 26826 raft_consensus.cc:3037] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:04.069123 26826 raft_consensus.cc:504] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857019d5412a4a529869fe7bc376be5f" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40257 } } peers { permanent_uuid: "dd1be0e0c4d2471bb2ae0b576f125e4b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34717 } } peers { permanent_uuid: "6c5c0b97905c4e7eab268e0ba7365259" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 46609 } }
I20260806 15:59:04.069483 26826 leader_election.cc:288] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [CANDIDATE]: Term 1 election: Requested vote from peers 857019d5412a4a529869fe7bc376be5f (127.25.31.131:40257), dd1be0e0c4d2471bb2ae0b576f125e4b (127.25.31.129:34717)
I20260806 15:59:04.070109 26755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0969d36de2643a19773325ae32cc208" candidate_uuid: "6c5c0b97905c4e7eab268e0ba7365259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "857019d5412a4a529869fe7bc376be5f"
I20260806 15:59:04.070102 26479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0969d36de2643a19773325ae32cc208" candidate_uuid: "6c5c0b97905c4e7eab268e0ba7365259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd1be0e0c4d2471bb2ae0b576f125e4b"
I20260806 15:59:04.070256 26479 raft_consensus.cc:3037] T e0969d36de2643a19773325ae32cc208 P dd1be0e0c4d2471bb2ae0b576f125e4b [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:04.070242 26755 raft_consensus.cc:3037] T e0969d36de2643a19773325ae32cc208 P 857019d5412a4a529869fe7bc376be5f [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:04.071314 26755 raft_consensus.cc:2454] T e0969d36de2643a19773325ae32cc208 P 857019d5412a4a529869fe7bc376be5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c5c0b97905c4e7eab268e0ba7365259 in term 1.
I20260806 15:59:04.071347 26479 raft_consensus.cc:2454] T e0969d36de2643a19773325ae32cc208 P dd1be0e0c4d2471bb2ae0b576f125e4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c5c0b97905c4e7eab268e0ba7365259 in term 1.
I20260806 15:59:04.071661 26549 leader_election.cc:302] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c5c0b97905c4e7eab268e0ba7365259, 857019d5412a4a529869fe7bc376be5f; no voters:
I20260806 15:59:04.071904 26826 raft_consensus.cc:2781] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:04.072104 26826 raft_consensus.cc:686] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [term 1 LEADER]: Becoming Leader. State: Replica: 6c5c0b97905c4e7eab268e0ba7365259, State: Running, Role: LEADER
I20260806 15:59:04.072427 26826 consensus_queue.cc:237] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857019d5412a4a529869fe7bc376be5f" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40257 } } peers { permanent_uuid: "dd1be0e0c4d2471bb2ae0b576f125e4b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34717 } } peers { permanent_uuid: "6c5c0b97905c4e7eab268e0ba7365259" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 46609 } }
I20260806 15:59:04.075618 26325 catalog_manager.cc:5495] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c5c0b97905c4e7eab268e0ba7365259 (127.25.31.130). New cstate: current_term: 1 leader_uuid: "6c5c0b97905c4e7eab268e0ba7365259" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "857019d5412a4a529869fe7bc376be5f" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd1be0e0c4d2471bb2ae0b576f125e4b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c5c0b97905c4e7eab268e0ba7365259" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 46609 } health_report { overall_health: HEALTHY } } }
W20260806 15:59:04.088557 26803 tablet.cc:2369] T e0969d36de2643a19773325ae32cc208 P 857019d5412a4a529869fe7bc376be5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 15:59:04 dist-test-slave-tm1g krb5kdc[26279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.129@KRBTEST.COM
W20260806 15:59:04.121838 26666 tablet.cc:2369] T e0969d36de2643a19773325ae32cc208 P 6c5c0b97905c4e7eab268e0ba7365259: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:59:04.123991 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:04.114503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49966 (local address 127.25.31.129:34717)
0806 15:59:04.114671 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:04.114676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:04.114800 (+ 124us) server_negotiation.cc:407] Connection header received
0806 15:59:04.114925 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:04.114927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:04.114970 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:04.115040 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:04.115826 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:04.116532 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:04.117398 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:04.117635 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:04.117735 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:04.120337 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:04.120357 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:04.120376 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:04.120402 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:04.122088 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:04.122632 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:04.122637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:04.122639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:04.122735 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:04.122985 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:04.122987 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:04.122988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:04.123268 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:04.123445 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:04.123619 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 15:59:04.123732 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 15:59:04 dist-test-slave-tm1g krb5kdc[26279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031944, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 15:59:04 dist-test-slave-tm1g krb5kdc[26279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031944, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.31.129@KRBTEST.COM
I20260806 15:59:04.152719 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:04.143349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49976 (local address 127.25.31.129:34717)
0806 15:59:04.143524 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:04.143528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:04.143603 (+ 75us) server_negotiation.cc:407] Connection header received
0806 15:59:04.143781 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:04.143784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:04.143852 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:04.143959 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:04.144773 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:04.145327 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:04.146560 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:04.146802 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:04.146902 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:04.149539 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:04.149542 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:04.149544 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:04.149569 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:04.150992 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:04.151418 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:04.151423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:04.151425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:04.151507 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:04.151783 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:04.151787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:04.151789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:04.151983 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:04.152120 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:04.152418 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 15:59:04.152548 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 15:59:04 dist-test-slave-tm1g krb5kdc[26279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 15:59:04 dist-test-slave-tm1g krb5kdc[26279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.129@KRBTEST.COM
I20260806 15:59:04.179879 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:04.171332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49992 (local address 127.25.31.129:34717)
0806 15:59:04.171482 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:04.171487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:04.171651 (+ 164us) server_negotiation.cc:407] Connection header received
0806 15:59:04.171833 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:04.171835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:04.171887 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:04.171971 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:04.172649 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:04.173165 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:04.174012 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:04.174233 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:04.174377 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:04.176739 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:04.176742 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:04.176744 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:04.176766 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:04.178141 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:04.178653 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:04.178656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:04.178658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:04.178738 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:04.179018 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:04.179022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:04.179023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:04.179201 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:04.179340 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:04.179581 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 15:59:04.179700 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 15:59:04.181318 26437 tablet_service.cc:2908] Rejecting scan request for tablet e0969d36de2643a19773325ae32cc208: Uninitialized: clean time has not yet been initialized
I20260806 15:59:04.183557 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 26392
I20260806 15:59:04.190398 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 26532
I20260806 15:59:04.196446 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 26670
I20260806 15:59:04.202170 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 26292
2026-08-06T15:59:04Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2409 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:59:04 dist-test-slave-tm1g krb5kdc[26844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:59:04 dist-test-slave-tm1g krb5kdc[26844](info): set up 2 sockets
Aug 06 15:59:04 dist-test-slave-tm1g krb5kdc[26844](info): commencing operation
krb5kdc: starting...
W20260806 15:59:05.258623 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 15:59:05 dist-test-slave-tm1g krb5kdc[26844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:59:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 15:59:05.385277 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:43381
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:43381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:05.494820 26858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:05.495244 26858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:05.495323 26858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:05.499409 26858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:05.499468 26858 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 15:59:05.499486 26858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:05.499504 26858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:05.499521 26858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:05.499575 26858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:05.504923 26858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:43381
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:43381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:05.506003 26858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:05.507145 26858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:05.515710 26866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:05.515754 26858 server_base.cc:1034] running on GCE node
W20260806 15:59:05.515726 26864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:05.515751 26863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:05.516523 26858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:05.517584 26858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:05.518769 26858 hybrid_clock.cc:648] HybridClock initialized: now 1786031945518756 us; error 26 us; skew 500 ppm
Aug 06 15:59:05 dist-test-slave-tm1g krb5kdc[26844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031945, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:05.521590 26858 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 15:59:05.522552 26858 webserver.cc:460] Webserver started at http://127.25.31.190:33463/ using document root <none> and password file <none>
I20260806 15:59:05.523097 26858 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:05.523187 26858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:05.523424 26858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:05.525556 26858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "973d336e1782420a86a1c6731f9f2e56"
format_stamp: "Formatted at 2026-08-06 15:59:05 on dist-test-slave-tm1g"
server_key: "246d11167566dc988a6c867bbd534b33"
server_key_iv: "9db0a0fed055ded167dcc3b40c9ea235"
server_key_version: "encryptionkey@0"
I20260806 15:59:05.526124 26858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "973d336e1782420a86a1c6731f9f2e56"
format_stamp: "Formatted at 2026-08-06 15:59:05 on dist-test-slave-tm1g"
server_key: "246d11167566dc988a6c867bbd534b33"
server_key_iv: "9db0a0fed055ded167dcc3b40c9ea235"
server_key_version: "encryptionkey@0"
I20260806 15:59:05.529803 26858 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 15:59:05.532204 26873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:05.533362 26858 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:05.533514 26858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "973d336e1782420a86a1c6731f9f2e56"
format_stamp: "Formatted at 2026-08-06 15:59:05 on dist-test-slave-tm1g"
server_key: "246d11167566dc988a6c867bbd534b33"
server_key_iv: "9db0a0fed055ded167dcc3b40c9ea235"
server_key_version: "encryptionkey@0"
I20260806 15:59:05.533622 26858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:05.554006 26858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:05.557232 26858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:05.557437 26858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:05.565564 26858 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:43381
I20260806 15:59:05.565582 26938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:43381 every 8 connection(s)
I20260806 15:59:05.566496 26858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:05.569159 26939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:05.571419 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 26858
I20260806 15:59:05.571549 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:05.571810 25726 external_mini_cluster.cc:1428] Setting key 0e473b3c5f4cf6b2a046ac5197796119
I20260806 15:59:05.575086 26939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56: Bootstrap starting.
I20260806 15:59:05.577214 26939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:05.577952 26939 log.cc:824] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:05.579828 26939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56: No bootstrap required, opened a new log
Aug 06 15:59:05 dist-test-slave-tm1g krb5kdc[26844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:05.584112 26939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "973d336e1782420a86a1c6731f9f2e56" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43381 } }
I20260806 15:59:05.584407 26939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:05.584484 26939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 973d336e1782420a86a1c6731f9f2e56, State: Initialized, Role: FOLLOWER
I20260806 15:59:05.584961 26939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "973d336e1782420a86a1c6731f9f2e56" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43381 } }
I20260806 15:59:05.585099 26939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:05.585191 26939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:05.585317 26939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:05.585837 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:05.573157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51384 (local address 127.25.31.190:43381)
0806 15:59:05.573593 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:05.573607 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:05.573641 (+ 34us) server_negotiation.cc:407] Connection header received
0806 15:59:05.574262 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:05.574277 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:05.574666 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:05.574989 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:05.575768 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:05.576767 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:05.577687 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:05.578201 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:05.578400 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:05.581156 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:05.581161 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:05.581173 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:05.581198 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:05.582857 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:05.583352 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:05.583357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:05.583361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:05.583477 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:05.583764 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:05.583767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:05.583768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:05.584119 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:05.584286 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:05.584677 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 15:59:05.584994 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":135,"threads_started":1}
I20260806 15:59:05.586261 26939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "973d336e1782420a86a1c6731f9f2e56" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43381 } }
I20260806 15:59:05.586647 26939 leader_election.cc:302] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 973d336e1782420a86a1c6731f9f2e56; no voters:
I20260806 15:59:05.586977 26939 leader_election.cc:288] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:05.587086 26944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:05.587311 26944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [term 1 LEADER]: Becoming Leader. State: Replica: 973d336e1782420a86a1c6731f9f2e56, State: Running, Role: LEADER
I20260806 15:59:05.587646 26944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "973d336e1782420a86a1c6731f9f2e56" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43381 } }
I20260806 15:59:05.588223 26939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:05.589455 26946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 973d336e1782420a86a1c6731f9f2e56. Latest consensus state: current_term: 1 leader_uuid: "973d336e1782420a86a1c6731f9f2e56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "973d336e1782420a86a1c6731f9f2e56" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43381 } } }
I20260806 15:59:05.589581 26946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:05.589871 26945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "973d336e1782420a86a1c6731f9f2e56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "973d336e1782420a86a1c6731f9f2e56" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43381 } } }
I20260806 15:59:05.589984 26953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:05.589972 26945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:05.593117 26953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:05.598650 26953 catalog_manager.cc:1347] Generated new cluster ID: d6452cd0349f489d98858ea472bef74c
I20260806 15:59:05.598722 26953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:05.606478 26953 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:05.607453 26953 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:05.614557 26953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56: Generated new TSK 0
I20260806 15:59:05.615211 26953 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 15:59:05.648559 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:43381
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:05.762238 26965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:05.762555 26965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:05.762631 26965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:05.766671 26965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:05.766723 26965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:05.766790 26965 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:59:05.766877 26965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:05.771967 26965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.31.190:43381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:05.772991 26965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:05.773972 26965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:05.782981 26971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:05.783092 26970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:05.783102 26965 server_base.cc:1034] running on GCE node
W20260806 15:59:05.783120 26973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:05.783677 26965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:05.784281 26965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:05.785478 26965 hybrid_clock.cc:648] HybridClock initialized: now 1786031945785461 us; error 26 us; skew 500 ppm
Aug 06 15:59:05 dist-test-slave-tm1g krb5kdc[26844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031945, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:05.788388 26965 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 15:59:05.789317 26965 webserver.cc:460] Webserver started at http://127.25.31.129:38191/ using document root <none> and password file <none>
I20260806 15:59:05.789939 26965 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:05.790004 26965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:05.790230 26965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:05.792347 26965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "2cff7af6dbc94e638325d0478336439a"
format_stamp: "Formatted at 2026-08-06 15:59:05 on dist-test-slave-tm1g"
server_key: "9364527670e8a838122aad3226a51697"
server_key_iv: "390e1c0f3e2b35f1328e388898d53256"
server_key_version: "encryptionkey@0"
I20260806 15:59:05.792958 26965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "2cff7af6dbc94e638325d0478336439a"
format_stamp: "Formatted at 2026-08-06 15:59:05 on dist-test-slave-tm1g"
server_key: "9364527670e8a838122aad3226a51697"
server_key_iv: "390e1c0f3e2b35f1328e388898d53256"
server_key_version: "encryptionkey@0"
I20260806 15:59:05.796723 26965 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 15:59:05.799183 26981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:05.800386 26965 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 15:59:05.800520 26965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "2cff7af6dbc94e638325d0478336439a"
format_stamp: "Formatted at 2026-08-06 15:59:05 on dist-test-slave-tm1g"
server_key: "9364527670e8a838122aad3226a51697"
server_key_iv: "390e1c0f3e2b35f1328e388898d53256"
server_key_version: "encryptionkey@0"
I20260806 15:59:05.800621 26965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:05.820827 26965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:05.823773 26965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:05.823990 26965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:05.825531 26965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:05.825580 26965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:05.825644 26965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:05.825675 26965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 15:59:05 dist-test-slave-tm1g krb5kdc[26844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031945, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:05.836846 26965 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:41073
I20260806 15:59:05.836870 27100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:41073 every 8 connection(s)
I20260806 15:59:05.837934 26965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:05.840835 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:05.826382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:54997 (local address 127.25.31.190:43381)
0806 15:59:05.826529 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:05.826534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:05.827414 (+ 880us) server_negotiation.cc:407] Connection header received
0806 15:59:05.828392 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:05.828396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:05.828454 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:05.828516 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:05.830395 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:05.831316 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:05.832572 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:05.832822 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:05.832945 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:05.836023 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:05.836026 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:05.836028 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:05.836052 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:05.838959 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:05.839427 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:05.839429 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:05.839430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:05.839504 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:05.839792 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:05.839794 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:05.839795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:05.839965 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:05.840041 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:05.840607 (+ 566us) server_negotiation.cc:299] Negotiation successful
0806 15:59:05.840718 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 15:59:05.841576 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:05.826801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:43381 (local address 127.25.31.129:54997)
0806 15:59:05.827244 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:05.827293 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:05.828141 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:05.829005 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:05.829013 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:05.829551 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:05.830197 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:05.830219 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:05.831469 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:05.831473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:05.832449 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:05.832463 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:05.832997 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:05.833001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:05.833179 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:05.833775 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:05.833793 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:05.835851 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:05.839066 (+ 3215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:05.839070 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:05.839079 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:05.839323 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:05.839580 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:05.839582 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:05.839583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:05.839709 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:05.840086 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:05.840091 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:05.840329 (+ 238us) client_negotiation.cc:769] Sending connection context
0806 15:59:05.840494 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 15:59:05.840731 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":131,"threads_started":1}
I20260806 15:59:05.842904 27101 heartbeater.cc:347] Connected to a master server at 127.25.31.190:43381
I20260806 15:59:05.843065 27101 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:05.843643 27101 heartbeater.cc:511] Master 127.25.31.190:43381 requested a full tablet report, sending...
I20260806 15:59:05.844748 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 26965
I20260806 15:59:05.844874 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 15:59:05.845207 25726 external_mini_cluster.cc:1428] Setting key b94e785c5ac28212380087180c8f3cbd
I20260806 15:59:05.845293 26892 ts_manager.cc:194] Registered new tserver with Master: 2cff7af6dbc94e638325d0478336439a (127.25.31.129:41073)
I20260806 15:59:05.846889 26892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:54997
I20260806 15:59:05.852706 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:05.845322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:53161 (local address 127.25.31.190:43381)
0806 15:59:05.845706 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:05.845712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:05.845743 (+ 31us) server_negotiation.cc:407] Connection header received
0806 15:59:05.845780 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:05.845782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:05.845827 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:05.845905 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:05.846421 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:05.847126 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:05.848130 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:05.848525 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:05.848627 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:05.849659 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:05.849662 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:05.849665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:05.849688 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:05.851015 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:05.851538 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:05.851540 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:05.851541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:05.851611 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:05.851903 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:05.851908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:05.851910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:05.852077 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:05.852188 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:05.852446 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 15:59:05.852555 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302}
I20260806 15:59:05.852717 27107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:05.845246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:43381 (local address 127.25.31.129:53161)
0806 15:59:05.845581 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:05.845596 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:05.845678 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:05.845938 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:05.845941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:05.846151 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:05.846294 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:05.846302 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:05.847276 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:05.847279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:05.848002 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:05.848013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:05.848634 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:05.848638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:05.848748 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:05.849202 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:05.849212 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:05.849518 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:05.851137 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:05.851140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:05.851142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:05.851346 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:05.851710 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:05.851714 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:05.851716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:05.851812 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:05.852178 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:05.852181 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:05.852280 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 15:59:05.852391 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 15:59:05.852530 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":151,"threads_started":1}
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 15:59:05.873793 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:43381
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:05.985031 27110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:05.985335 27110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:05.985445 27110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:05.989734 27110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:05.989809 27110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:05.989874 27110 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:59:05.989959 27110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:05.995244 27110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.31.190:43381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:05.996249 27110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:05.997257 27110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:06.006699 27117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:06.006699 27116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:06.006706 27119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:06.006976 27110 server_base.cc:1034] running on GCE node
I20260806 15:59:06.007431 27110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:06.008145 27110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:06.009384 27110 hybrid_clock.cc:648] HybridClock initialized: now 1786031946009366 us; error 35 us; skew 500 ppm
Aug 06 15:59:06 dist-test-slave-tm1g krb5kdc[26844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031946, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:06.012244 27110 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 15:59:06.013079 27110 webserver.cc:460] Webserver started at http://127.25.31.130:41149/ using document root <none> and password file <none>
I20260806 15:59:06.013629 27110 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:06.013688 27110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:06.013855 27110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:06.015923 27110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "4d8dac4393df49b5a7a17d5d844bd538"
format_stamp: "Formatted at 2026-08-06 15:59:06 on dist-test-slave-tm1g"
server_key: "bf1f76fbdde4c40d9baf62cc00c1bdb4"
server_key_iv: "a9572c3c2ec557d0411b0563e02f7437"
server_key_version: "encryptionkey@0"
I20260806 15:59:06.016450 27110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "4d8dac4393df49b5a7a17d5d844bd538"
format_stamp: "Formatted at 2026-08-06 15:59:06 on dist-test-slave-tm1g"
server_key: "bf1f76fbdde4c40d9baf62cc00c1bdb4"
server_key_iv: "a9572c3c2ec557d0411b0563e02f7437"
server_key_version: "encryptionkey@0"
I20260806 15:59:06.020125 27110 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 15:59:06.022464 27126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:06.023643 27110 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:06.023762 27110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "4d8dac4393df49b5a7a17d5d844bd538"
format_stamp: "Formatted at 2026-08-06 15:59:06 on dist-test-slave-tm1g"
server_key: "bf1f76fbdde4c40d9baf62cc00c1bdb4"
server_key_iv: "a9572c3c2ec557d0411b0563e02f7437"
server_key_version: "encryptionkey@0"
I20260806 15:59:06.023860 27110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:06.039390 27110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:06.042063 27110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:06.042253 27110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:06.043780 27110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:06.043829 27110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:06.043895 27110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:06.043927 27110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 15:59:06 dist-test-slave-tm1g krb5kdc[26844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031946, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:06.054603 27110 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:39589
I20260806 15:59:06.054646 27246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:39589 every 8 connection(s)
I20260806 15:59:06.055590 27110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:06.058451 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.044588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:55285 (local address 127.25.31.190:43381)
0806 15:59:06.044738 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:06.044743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:06.045626 (+ 883us) server_negotiation.cc:407] Connection header received
0806 15:59:06.046810 (+ 1184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:06.046813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:06.046873 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:06.046957 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:06.048421 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.049349 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.051305 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.051660 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.051744 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.054752 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:06.054755 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:06.054758 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:06.054784 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:06.056365 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:06.056911 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:06.056914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:06.056916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:06.056999 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:06.057336 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:06.057339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:06.057341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:06.057528 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:06.057635 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:06.058122 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 15:59:06.058250 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 15:59:06.059163 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.045017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:43381 (local address 127.25.31.130:55285)
0806 15:59:06.045465 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:06.045514 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:06.046550 (+ 1036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:06.047154 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:06.047163 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:06.047643 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:06.048275 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.048297 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.049515 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.049520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.051159 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.051175 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.051819 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.051823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.052057 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.052725 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:06.052749 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:06.054571 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:06.056466 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:06.056471 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:06.056485 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:06.056800 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:06.057091 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:06.057095 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:06.057097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:06.057233 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:06.057635 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:06.057642 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:06.057870 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 15:59:06.058062 (+ 192us) client_negotiation.cc:240] Negotiation successful
0806 15:59:06.058391 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":127,"threads_started":1}
I20260806 15:59:06.060071 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 27110
I20260806 15:59:06.060180 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 15:59:06.060374 27247 heartbeater.cc:347] Connected to a master server at 127.25.31.190:43381
I20260806 15:59:06.060473 25726 external_mini_cluster.cc:1428] Setting key 95355cd1f7ceee27b18548e62aeb979e
I20260806 15:59:06.060526 27247 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:06.061075 27247 heartbeater.cc:511] Master 127.25.31.190:43381 requested a full tablet report, sending...
I20260806 15:59:06.062103 26891 ts_manager.cc:194] Registered new tserver with Master: 4d8dac4393df49b5a7a17d5d844bd538 (127.25.31.130:39589)
I20260806 15:59:06.062901 26891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:55285
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
I20260806 15:59:06.070258 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.062822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:52685 (local address 127.25.31.190:43381)
0806 15:59:06.063214 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:06.063218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:06.063229 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:59:06.063257 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:06.063260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:06.063300 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:06.063383 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:06.063895 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.064618 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.065833 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.066043 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.066126 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.067262 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:06.067265 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:06.067268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:06.067291 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:06.068672 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:06.069085 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:06.069089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:06.069091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:06.069167 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:06.069440 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:06.069445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:06.069446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:06.069648 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:06.069755 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:06.069998 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 15:59:06.070105 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316}
I20260806 15:59:06.070333 27253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.062727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:43381 (local address 127.25.31.130:52685)
0806 15:59:06.063038 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:06.063052 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:06.063136 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:06.063399 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:06.063402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:06.063606 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:06.063761 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.063768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.064778 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.064780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.065714 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.065728 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.066160 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.066166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.066319 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.066826 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:06.066845 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:06.067126 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:06.068790 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:06.068795 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:06.068797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:06.068988 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:06.069257 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:06.069261 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:06.069263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:06.069346 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:06.069744 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:06.069747 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:06.069841 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 15:59:06.069945 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 15:59:06.070085 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":16,"thread_start_us":88,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 15:59:06.088665 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:43381
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:06.200573 27257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:06.200901 27257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:06.200963 27257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:06.205336 27257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:06.205394 27257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:06.205463 27257 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:59:06.205549 27257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:06.210899 27257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.31.190:43381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:06.211874 27257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:06.212839 27257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:06.222354 27265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:06.222378 27263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:06.222440 27257 server_base.cc:1034] running on GCE node
W20260806 15:59:06.222397 27262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:06.223042 27257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:06.223627 27257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:06.224802 27257 hybrid_clock.cc:648] HybridClock initialized: now 1786031946224775 us; error 41 us; skew 500 ppm
Aug 06 15:59:06 dist-test-slave-tm1g krb5kdc[26844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031946, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:06.227435 27257 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 15:59:06.228366 27257 webserver.cc:460] Webserver started at http://127.25.31.131:41331/ using document root <none> and password file <none>
I20260806 15:59:06.228899 27257 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:06.228988 27257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:06.229223 27257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:06.231235 27257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "f9f40b680bf440b7a7b849af709ee4f5"
format_stamp: "Formatted at 2026-08-06 15:59:06 on dist-test-slave-tm1g"
server_key: "5a8ef7eb22771f141b6ac3b0a4e57ad2"
server_key_iv: "8aa198dbe4c2ad6622be16c01f8231b2"
server_key_version: "encryptionkey@0"
I20260806 15:59:06.231788 27257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "f9f40b680bf440b7a7b849af709ee4f5"
format_stamp: "Formatted at 2026-08-06 15:59:06 on dist-test-slave-tm1g"
server_key: "5a8ef7eb22771f141b6ac3b0a4e57ad2"
server_key_iv: "8aa198dbe4c2ad6622be16c01f8231b2"
server_key_version: "encryptionkey@0"
I20260806 15:59:06.235322 27257 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 15:59:06.237851 27272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:06.239097 27257 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:06.239234 27257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "f9f40b680bf440b7a7b849af709ee4f5"
format_stamp: "Formatted at 2026-08-06 15:59:06 on dist-test-slave-tm1g"
server_key: "5a8ef7eb22771f141b6ac3b0a4e57ad2"
server_key_iv: "8aa198dbe4c2ad6622be16c01f8231b2"
server_key_version: "encryptionkey@0"
I20260806 15:59:06.239341 27257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:06.251024 27257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:06.253777 27257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:06.253968 27257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:06.255450 27257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:06.255522 27257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:06.255584 27257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:06.255633 27257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 15:59:06 dist-test-slave-tm1g krb5kdc[26844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031946, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:06.266973 27257 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:35839
I20260806 15:59:06.267055 27389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:35839 every 8 connection(s)
I20260806 15:59:06.267807 27257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 15:59:06.269395 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.256288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:34337 (local address 127.25.31.190:43381)
0806 15:59:06.256424 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:06.256428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:06.257450 (+ 1022us) server_negotiation.cc:407] Connection header received
0806 15:59:06.258411 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:06.258418 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:06.258490 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:06.258604 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:06.259862 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.260763 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.262278 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.262625 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.262735 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.265531 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:06.265537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:06.265541 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:06.265574 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:06.267286 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:06.267814 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:06.267817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:06.267819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:06.267918 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:06.268260 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:06.268266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:06.268267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:06.268458 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:06.268565 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:06.269102 (+ 537us) server_negotiation.cc:299] Negotiation successful
0806 15:59:06.269223 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 15:59:06.270123 27282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.256811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:43381 (local address 127.25.31.131:34337)
0806 15:59:06.257288 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:06.257330 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:06.258150 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:06.258927 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:06.258933 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:06.259234 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:06.259694 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.259707 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.260889 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.260896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.262143 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.262154 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.262791 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.262796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.262969 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.263568 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:06.263586 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:06.265360 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:06.267405 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:06.267411 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:06.267425 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:06.267694 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:06.268014 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:06.268018 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:06.268021 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:06.268158 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:06.268566 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:06.268573 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:06.268866 (+ 293us) client_negotiation.cc:769] Sending connection context
0806 15:59:06.269058 (+ 192us) client_negotiation.cc:240] Negotiation successful
0806 15:59:06.269366 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":130,"threads_started":1}
I20260806 15:59:06.271325 27390 heartbeater.cc:347] Connected to a master server at 127.25.31.190:43381
I20260806 15:59:06.271458 27390 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:06.271999 27390 heartbeater.cc:511] Master 127.25.31.190:43381 requested a full tablet report, sending...
I20260806 15:59:06.273072 26892 ts_manager.cc:194] Registered new tserver with Master: f9f40b680bf440b7a7b849af709ee4f5 (127.25.31.131:35839)
I20260806 15:59:06.273746 26892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:34337
I20260806 15:59:06.274933 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 27257
I20260806 15:59:06.275075 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 15:59:06.275329 25726 external_mini_cluster.cc:1428] Setting key 70a4ddc1085d353e3140e99a8ecf50f8
I20260806 15:59:06.277320 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:59:06.280857 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.273590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:58197 (local address 127.25.31.190:43381)
0806 15:59:06.273714 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:06.273718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:06.273908 (+ 190us) server_negotiation.cc:407] Connection header received
0806 15:59:06.274042 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:06.274045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:06.274092 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:06.274170 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:06.274663 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.275427 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.276529 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.276734 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.276844 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.277920 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:06.277923 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:06.277926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:06.277949 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:06.279267 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:06.279703 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:06.279706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:06.279708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:06.279789 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:06.280069 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:06.280074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:06.280075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:06.280244 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:06.280361 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:06.280607 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 15:59:06.280715 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 15:59:06.280884 27396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.273516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:43381 (local address 127.25.31.131:58197)
0806 15:59:06.273835 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:06.273851 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:06.273943 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:06.274202 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:06.274205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:06.274444 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:06.274550 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.274555 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.275573 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.275576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.276397 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.276407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.276871 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.276875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.276976 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.277483 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:06.277494 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:06.277781 (+ 287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:06.279398 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:06.279401 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:06.279403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:06.279599 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:06.279889 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:06.279892 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:06.279894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:06.279981 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:06.280343 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:06.280347 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:06.280437 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 15:59:06.280551 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 15:59:06.280691 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":111,"threads_started":1}
Aug 06 15:59:06 dist-test-slave-tm1g krb5kdc[26844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031946, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 15:59:06 dist-test-slave-tm1g krb5kdc[26844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031946, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:06.303586 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.295352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51396 (local address 127.25.31.190:43381)
0806 15:59:06.295472 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:06.295482 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:06.295494 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:59:06.295532 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:06.295535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:06.295571 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:06.295664 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:06.296371 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.296902 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.297758 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.297982 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.298089 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.300554 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:06.300557 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:06.300559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:06.300583 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:06.301873 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:06.302332 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:06.302336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:06.302338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:06.302436 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:06.302724 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:06.302730 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:06.302732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:06.302913 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:06.303056 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:06.303289 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 15:59:06.303415 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 15:59:06.306671 26892 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:59:06.309801 26892 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:06.324563 27410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.318502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:39589 (local address 127.0.0.1:40432)
0806 15:59:06.319495 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:06.319508 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:06.319582 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:06.320241 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:06.320244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:06.320258 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:06.320348 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.320356 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.321696 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.321700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.322925 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.322943 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.324016 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.324019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.324120 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.324147 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 15:59:06.324227 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 15:59:06.324414 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"mutex_wait_us":4,"thread_start_us":92,"threads_started":1}
I20260806 15:59:06.324542 27412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.318890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.25.31.131:35839)
0806 15:59:06.319314 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:06.319325 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:06.319349 (+ 24us) server_negotiation.cc:407] Connection header received
0806 15:59:06.319582 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:06.319588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:06.319801 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:06.319950 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:06.320374 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.321589 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.323163 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.323735 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.323855 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.324031 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:06.324297 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 15:59:06.324393 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":119,"threads_started":1}
I20260806 15:59:06.324801 27409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.318251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59662 (local address 127.25.31.129:41073)
0806 15:59:06.318882 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:06.318898 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:06.319467 (+ 569us) server_negotiation.cc:407] Connection header received
0806 15:59:06.319616 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:06.319620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:06.319792 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:06.319950 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:06.320461 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.321201 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.322787 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.323547 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.323652 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.323816 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:06.324344 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 15:59:06.324437 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":122,"threads_started":1}
I20260806 15:59:06.324965 27413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.318905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40432 (local address 127.25.31.130:39589)
0806 15:59:06.319793 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:06.319804 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:06.319821 (+ 17us) server_negotiation.cc:407] Connection header received
0806 15:59:06.319903 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:06.319910 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:06.320101 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:06.320268 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:06.320426 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.321552 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.323060 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.323885 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.324500 (+ 615us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.324661 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:06.324712 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 15:59:06.324807 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":116,"threads_started":1}
I20260806 15:59:06.325328 27408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.318094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:41073 (local address 127.0.0.1:59662)
0806 15:59:06.319354 (+ 1260us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:06.319369 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:06.319483 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:06.319963 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:06.319968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:06.319991 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:06.320125 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.320132 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.321321 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.321326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.322668 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.322677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.323885 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.323900 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.324077 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.324147 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 15:59:06.324243 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 15:59:06.325168 (+ 925us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":878,"thread_start_us":80,"threads_started":1}
I20260806 15:59:06.325503 27411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.318250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:35839 (local address 127.0.0.1:47234)
0806 15:59:06.319152 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:06.319208 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:06.319339 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:06.320107 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:06.320110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:06.320135 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:06.320247 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.320253 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.321909 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.321912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.323046 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.323053 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.323857 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.323860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.324104 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.324148 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 15:59:06.324263 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 15:59:06.324366 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":81,"thread_start_us":126,"threads_started":1}
I20260806 15:59:06.327986 27323 tablet_service.cc:1467] Processing CreateTablet for tablet 35289bcb1af3446bae6522da59ed88be (DEFAULT_TABLE table=test-table [id=7f5cf37737d548b191c3befc3643b215]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:06.328011 27178 tablet_service.cc:1467] Processing CreateTablet for tablet 35289bcb1af3446bae6522da59ed88be (DEFAULT_TABLE table=test-table [id=7f5cf37737d548b191c3befc3643b215]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:06.328011 27032 tablet_service.cc:1467] Processing CreateTablet for tablet 35289bcb1af3446bae6522da59ed88be (DEFAULT_TABLE table=test-table [id=7f5cf37737d548b191c3befc3643b215]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:06.329006 27323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35289bcb1af3446bae6522da59ed88be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:06.329021 27178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35289bcb1af3446bae6522da59ed88be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:06.329021 27032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35289bcb1af3446bae6522da59ed88be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:06.335440 27414 tablet_bootstrap.cc:492] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5: Bootstrap starting.
I20260806 15:59:06.337148 27415 tablet_bootstrap.cc:492] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538: Bootstrap starting.
I20260806 15:59:06.337741 27414 tablet_bootstrap.cc:654] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:06.338639 27416 tablet_bootstrap.cc:492] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a: Bootstrap starting.
I20260806 15:59:06.338727 27414 log.cc:824] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:06.339077 27415 tablet_bootstrap.cc:654] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:06.339915 27415 log.cc:824] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:06.340744 27416 tablet_bootstrap.cc:654] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:06.340864 27414 tablet_bootstrap.cc:492] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5: No bootstrap required, opened a new log
I20260806 15:59:06.341063 27414 ts_tablet_manager.cc:1397] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 15:59:06.341518 27416 log.cc:824] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:06.341908 27415 tablet_bootstrap.cc:492] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538: No bootstrap required, opened a new log
I20260806 15:59:06.342092 27415 ts_tablet_manager.cc:1397] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 15:59:06.343696 27416 tablet_bootstrap.cc:492] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a: No bootstrap required, opened a new log
I20260806 15:59:06.343926 27416 ts_tablet_manager.cc:1397] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 15:59:06.344733 27414 raft_consensus.cc:348] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } }
I20260806 15:59:06.344980 27414 raft_consensus.cc:374] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:06.345034 27414 raft_consensus.cc:729] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9f40b680bf440b7a7b849af709ee4f5, State: Initialized, Role: FOLLOWER
I20260806 15:59:06.345451 27414 consensus_queue.cc:260] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } }
I20260806 15:59:06.345928 27415 raft_consensus.cc:348] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } }
I20260806 15:59:06.346107 27390 heartbeater.cc:503] Master 127.25.31.190:43381 was elected leader, sending a full tablet report...
I20260806 15:59:06.346165 27415 raft_consensus.cc:374] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:06.346218 27415 raft_consensus.cc:729] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8dac4393df49b5a7a17d5d844bd538, State: Initialized, Role: FOLLOWER
I20260806 15:59:06.346549 27414 ts_tablet_manager.cc:1428] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260806 15:59:06.346661 27415 consensus_queue.cc:260] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } }
I20260806 15:59:06.347303 27247 heartbeater.cc:503] Master 127.25.31.190:43381 was elected leader, sending a full tablet report...
I20260806 15:59:06.347584 27415 ts_tablet_manager.cc:1428] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 15:59:06.348482 27416 raft_consensus.cc:348] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } }
I20260806 15:59:06.348719 27416 raft_consensus.cc:374] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:06.348779 27416 raft_consensus.cc:729] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cff7af6dbc94e638325d0478336439a, State: Initialized, Role: FOLLOWER
I20260806 15:59:06.349216 27416 consensus_queue.cc:260] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } }
I20260806 15:59:06.350152 27416 ts_tablet_manager.cc:1428] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 15:59:06.350272 27101 heartbeater.cc:503] Master 127.25.31.190:43381 was elected leader, sending a full tablet report...
I20260806 15:59:06.509805 27423 raft_consensus.cc:482] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:06.510002 27423 raft_consensus.cc:504] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } }
I20260806 15:59:06.511176 27423 leader_election.cc:288] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9f40b680bf440b7a7b849af709ee4f5 (127.25.31.131:35839), 4d8dac4393df49b5a7a17d5d844bd538 (127.25.31.130:39589)
I20260806 15:59:06.515353 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.511353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:35839 (local address 127.25.31.129:50369)
0806 15:59:06.511695 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:06.511712 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:06.511875 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:06.512195 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:06.512198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:06.512216 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:06.512383 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.512389 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.513176 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.513179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.514139 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.514149 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.514897 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.514901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.515004 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.515050 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 15:59:06.515109 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 15:59:06.515164 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":96,"spinlock_wait_cycles":21760,"thread_start_us":80,"threads_started":1}
I20260806 15:59:06.515374 27412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.511534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:50369 (local address 127.25.31.131:35839)
0806 15:59:06.511675 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:06.511680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:06.511780 (+ 100us) server_negotiation.cc:407] Connection header received
0806 15:59:06.512020 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:06.512027 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:06.512087 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:06.512148 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:06.512497 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.513053 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.514267 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.514749 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.514871 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.514931 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:06.515148 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 15:59:06.515237 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 15:59:06.515413 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.511473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:39589 (local address 127.25.31.129:41695)
0806 15:59:06.511842 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:06.511854 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:06.511964 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:06.512229 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:06.512232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:06.512263 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:06.512413 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.512419 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.513217 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.513220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.514157 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.514167 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.514931 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.514935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.515087 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.515137 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 15:59:06.515215 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 15:59:06.515280 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":89,"thread_start_us":67,"threads_started":1}
I20260806 15:59:06.515463 27413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.511700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:41695 (local address 127.25.31.130:39589)
0806 15:59:06.511834 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:06.511838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:06.511934 (+ 96us) server_negotiation.cc:407] Connection header received
0806 15:59:06.512058 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:06.512060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:06.512099 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:06.512193 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:06.512561 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.513068 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.514369 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.514776 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.514921 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.514973 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:06.515255 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 15:59:06.515326 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 15:59:06.516014 27199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35289bcb1af3446bae6522da59ed88be" candidate_uuid: "2cff7af6dbc94e638325d0478336439a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8dac4393df49b5a7a17d5d844bd538" is_pre_election: true
I20260806 15:59:06.516018 27342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35289bcb1af3446bae6522da59ed88be" candidate_uuid: "2cff7af6dbc94e638325d0478336439a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9f40b680bf440b7a7b849af709ee4f5" is_pre_election: true
I20260806 15:59:06.516340 27199 raft_consensus.cc:2454] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cff7af6dbc94e638325d0478336439a in term 0.
I20260806 15:59:06.516340 27342 raft_consensus.cc:2454] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cff7af6dbc94e638325d0478336439a in term 0.
I20260806 15:59:06.516897 26985 leader_election.cc:302] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cff7af6dbc94e638325d0478336439a, f9f40b680bf440b7a7b849af709ee4f5; no voters:
I20260806 15:59:06.517160 27423 raft_consensus.cc:2781] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:06.517253 27423 raft_consensus.cc:482] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:06.517298 27423 raft_consensus.cc:3037] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:06.518428 27423 raft_consensus.cc:504] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } }
I20260806 15:59:06.518786 27423 leader_election.cc:288] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [CANDIDATE]: Term 1 election: Requested vote from peers f9f40b680bf440b7a7b849af709ee4f5 (127.25.31.131:35839), 4d8dac4393df49b5a7a17d5d844bd538 (127.25.31.130:39589)
W20260806 15:59:06.519145 27391 tablet.cc:2369] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:59:06.519143 27342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35289bcb1af3446bae6522da59ed88be" candidate_uuid: "2cff7af6dbc94e638325d0478336439a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9f40b680bf440b7a7b849af709ee4f5"
I20260806 15:59:06.519238 27199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35289bcb1af3446bae6522da59ed88be" candidate_uuid: "2cff7af6dbc94e638325d0478336439a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8dac4393df49b5a7a17d5d844bd538"
I20260806 15:59:06.519310 27342 raft_consensus.cc:3037] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:06.519413 27199 raft_consensus.cc:3037] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:06.520748 27342 raft_consensus.cc:2454] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cff7af6dbc94e638325d0478336439a in term 1.
I20260806 15:59:06.520852 27199 raft_consensus.cc:2454] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cff7af6dbc94e638325d0478336439a in term 1.
I20260806 15:59:06.521116 26985 leader_election.cc:302] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cff7af6dbc94e638325d0478336439a, f9f40b680bf440b7a7b849af709ee4f5; no voters:
I20260806 15:59:06.521330 27423 raft_consensus.cc:2781] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:06.521565 27423 raft_consensus.cc:686] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [term 1 LEADER]: Becoming Leader. State: Replica: 2cff7af6dbc94e638325d0478336439a, State: Running, Role: LEADER
I20260806 15:59:06.521931 27423 consensus_queue.cc:237] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } }
I20260806 15:59:06.525161 26892 catalog_manager.cc:5495] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cff7af6dbc94e638325d0478336439a (127.25.31.129). New cstate: current_term: 1 leader_uuid: "2cff7af6dbc94e638325d0478336439a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } health_report { overall_health: UNKNOWN } } }
W20260806 15:59:06.556826 27248 tablet.cc:2369] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:59:06.568600 27409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.564701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59670 (local address 127.25.31.129:41073)
0806 15:59:06.564875 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:06.564880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:06.564894 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:59:06.565130 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:06.565132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:06.565189 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:06.565312 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:06.565632 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.566165 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.567099 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.567332 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.567442 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.567796 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:06.568180 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:06.568308 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:06.568395 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 15:59:06.568455 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 15:59:06.574785 27199 raft_consensus.cc:1261] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538 [term 1 FOLLOWER]: Refusing update from remote peer 2cff7af6dbc94e638325d0478336439a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:06.574785 27342 raft_consensus.cc:1261] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5 [term 1 FOLLOWER]: Refusing update from remote peer 2cff7af6dbc94e638325d0478336439a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:06.575482 27423 consensus_queue.cc:1035] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:06.575675 27426 consensus_queue.cc:1035] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:06.583608 27430 mvcc.cc:205] Tried to move back new op lower bound from 7315586853162774528 to 7315586852956672000. Current Snapshot: MvccSnapshot[applied={T|T < 7315586853162774528}]
I20260806 15:59:06.595520 26892 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:06.595727 26892 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:06.598409 26892 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56: Sending DeleteTablet for 3 replicas of tablet 35289bcb1af3446bae6522da59ed88be
I20260806 15:59:06.599260 27178 tablet_service.cc:1514] Processing DeleteTablet for tablet 35289bcb1af3446bae6522da59ed88be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:06 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:40432
I20260806 15:59:06.599278 27323 tablet_service.cc:1514] Processing DeleteTablet for tablet 35289bcb1af3446bae6522da59ed88be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:06 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:47234
I20260806 15:59:06.599903 26892 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 15:59:06.600095 27032 tablet_service.cc:1514] Processing DeleteTablet for tablet 35289bcb1af3446bae6522da59ed88be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:06 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:59662
I20260806 15:59:06.600311 27442 tablet_replica.cc:330] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5: stopping tablet replica
W20260806 15:59:06.600376 26892 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:06.600682 27442 raft_consensus.cc:2229] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:06.601014 27442 raft_consensus.cc:2258] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:06.601428 27441 tablet_replica.cc:330] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538: stopping tablet replica
I20260806 15:59:06.601727 27441 raft_consensus.cc:2229] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:06.602912 27442 ts_tablet_manager.cc:1905] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:06.605854 27178 tablet_service.cc:1467] Processing CreateTablet for tablet e69a7828350442cb8b77a68370165c90 (DEFAULT_TABLE table=test-table [id=a9e93d72d070470c818bcf641638537e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:06.605897 27032 tablet_service.cc:1467] Processing CreateTablet for tablet e69a7828350442cb8b77a68370165c90 (DEFAULT_TABLE table=test-table [id=a9e93d72d070470c818bcf641638537e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:06.606137 27032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e69a7828350442cb8b77a68370165c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:06.606137 27178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e69a7828350442cb8b77a68370165c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:06.606149 27323 tablet_service.cc:1467] Processing CreateTablet for tablet e69a7828350442cb8b77a68370165c90 (DEFAULT_TABLE table=test-table [id=a9e93d72d070470c818bcf641638537e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:06.606427 27323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e69a7828350442cb8b77a68370165c90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:06.608634 27415 tablet_bootstrap.cc:492] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538: Bootstrap starting.
I20260806 15:59:06.608768 27416 tablet_bootstrap.cc:492] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a: Bootstrap starting.
I20260806 15:59:06.608839 27414 tablet_bootstrap.cc:492] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5: Bootstrap starting.
I20260806 15:59:06.609081 27442 ts_tablet_manager.cc:1918] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:06.609174 27442 log.cc:1196] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/35289bcb1af3446bae6522da59ed88be
I20260806 15:59:06.609512 27442 ts_tablet_manager.cc:1939] T 35289bcb1af3446bae6522da59ed88be P f9f40b680bf440b7a7b849af709ee4f5: Deleting consensus metadata
I20260806 15:59:06.609817 27414 tablet_bootstrap.cc:654] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:06.609865 27416 tablet_bootstrap.cc:654] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:06.611330 27415 tablet_bootstrap.cc:654] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:06.612494 26877 catalog_manager.cc:4841] TS f9f40b680bf440b7a7b849af709ee4f5 (127.25.31.131:35839): tablet 35289bcb1af3446bae6522da59ed88be (table test-table [id=7f5cf37737d548b191c3befc3643b215]) successfully deleted
I20260806 15:59:06.612684 27415 tablet_bootstrap.cc:492] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538: No bootstrap required, opened a new log
I20260806 15:59:06.612797 27415 ts_tablet_manager.cc:1397] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 15:59:06.613078 27441 raft_consensus.cc:2258] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:06.613350 27415 raft_consensus.cc:348] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.613510 27415 raft_consensus.cc:374] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:06.613551 27415 raft_consensus.cc:729] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8dac4393df49b5a7a17d5d844bd538, State: Initialized, Role: FOLLOWER
I20260806 15:59:06.613700 27415 consensus_queue.cc:260] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.614023 27415 ts_tablet_manager.cc:1428] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 15:59:06.614092 27414 tablet_bootstrap.cc:492] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5: No bootstrap required, opened a new log
I20260806 15:59:06.614167 27414 ts_tablet_manager.cc:1397] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260806 15:59:06.614658 27414 raft_consensus.cc:348] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.614773 27414 raft_consensus.cc:374] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:06.614823 27414 raft_consensus.cc:729] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9f40b680bf440b7a7b849af709ee4f5, State: Initialized, Role: FOLLOWER
I20260806 15:59:06.614948 27414 consensus_queue.cc:260] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.615221 27414 ts_tablet_manager.cc:1428] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 15:59:06.617841 27441 ts_tablet_manager.cc:1905] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:06.619145 27416 tablet_bootstrap.cc:492] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a: No bootstrap required, opened a new log
I20260806 15:59:06.619242 27416 ts_tablet_manager.cc:1397] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20260806 15:59:06.619637 27416 raft_consensus.cc:348] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.619722 27416 raft_consensus.cc:374] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:06.619743 27416 raft_consensus.cc:729] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cff7af6dbc94e638325d0478336439a, State: Initialized, Role: FOLLOWER
I20260806 15:59:06.619872 27416 consensus_queue.cc:260] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.620142 27416 ts_tablet_manager.cc:1428] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 15:59:06.620577 27441 ts_tablet_manager.cc:1918] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:06.620676 27441 log.cc:1196] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/35289bcb1af3446bae6522da59ed88be
I20260806 15:59:06.620990 27441 ts_tablet_manager.cc:1939] T 35289bcb1af3446bae6522da59ed88be P 4d8dac4393df49b5a7a17d5d844bd538: Deleting consensus metadata
I20260806 15:59:06.622117 26875 catalog_manager.cc:4841] TS 4d8dac4393df49b5a7a17d5d844bd538 (127.25.31.130:39589): tablet 35289bcb1af3446bae6522da59ed88be (table test-table [id=7f5cf37737d548b191c3befc3643b215]) successfully deleted
I20260806 15:59:06.622262 27443 tablet_replica.cc:330] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a: stopping tablet replica
I20260806 15:59:06.622625 27443 raft_consensus.cc:2229] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:59:06.623014 27443 raft_consensus.cc:2258] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:06.624260 27443 ts_tablet_manager.cc:1905] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:06.626902 27443 ts_tablet_manager.cc:1918] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:06.626999 27443 log.cc:1196] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/wal/wals/35289bcb1af3446bae6522da59ed88be
I20260806 15:59:06.627326 27443 ts_tablet_manager.cc:1939] T 35289bcb1af3446bae6522da59ed88be P 2cff7af6dbc94e638325d0478336439a: Deleting consensus metadata
I20260806 15:59:06.628491 26874 catalog_manager.cc:4841] TS 2cff7af6dbc94e638325d0478336439a (127.25.31.129:41073): tablet 35289bcb1af3446bae6522da59ed88be (table test-table [id=7f5cf37737d548b191c3befc3643b215]) successfully deleted
I20260806 15:59:06.662200 27439 raft_consensus.cc:482] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:06.662415 27439 raft_consensus.cc:504] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.662894 27439 leader_election.cc:288] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d8dac4393df49b5a7a17d5d844bd538 (127.25.31.130:39589), f9f40b680bf440b7a7b849af709ee4f5 (127.25.31.131:35839)
I20260806 15:59:06.663326 27342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e69a7828350442cb8b77a68370165c90" candidate_uuid: "2cff7af6dbc94e638325d0478336439a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9f40b680bf440b7a7b849af709ee4f5" is_pre_election: true
I20260806 15:59:06.663324 27199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e69a7828350442cb8b77a68370165c90" candidate_uuid: "2cff7af6dbc94e638325d0478336439a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8dac4393df49b5a7a17d5d844bd538" is_pre_election: true
I20260806 15:59:06.663486 27199 raft_consensus.cc:2454] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cff7af6dbc94e638325d0478336439a in term 0.
I20260806 15:59:06.663486 27342 raft_consensus.cc:2454] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cff7af6dbc94e638325d0478336439a in term 0.
I20260806 15:59:06.663790 26985 leader_election.cc:302] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cff7af6dbc94e638325d0478336439a, f9f40b680bf440b7a7b849af709ee4f5; no voters:
I20260806 15:59:06.664000 27439 raft_consensus.cc:2781] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:06.664062 27439 raft_consensus.cc:482] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:06.664103 27439 raft_consensus.cc:3037] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:06.665354 27439 raft_consensus.cc:504] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.665724 27439 leader_election.cc:288] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [CANDIDATE]: Term 1 election: Requested vote from peers 4d8dac4393df49b5a7a17d5d844bd538 (127.25.31.130:39589), f9f40b680bf440b7a7b849af709ee4f5 (127.25.31.131:35839)
I20260806 15:59:06.666051 27342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e69a7828350442cb8b77a68370165c90" candidate_uuid: "2cff7af6dbc94e638325d0478336439a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9f40b680bf440b7a7b849af709ee4f5"
I20260806 15:59:06.666051 27199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e69a7828350442cb8b77a68370165c90" candidate_uuid: "2cff7af6dbc94e638325d0478336439a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8dac4393df49b5a7a17d5d844bd538"
I20260806 15:59:06.666180 27199 raft_consensus.cc:3037] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:06.666180 27342 raft_consensus.cc:3037] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:06.667317 27199 raft_consensus.cc:2454] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cff7af6dbc94e638325d0478336439a in term 1.
I20260806 15:59:06.667317 27342 raft_consensus.cc:2454] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cff7af6dbc94e638325d0478336439a in term 1.
I20260806 15:59:06.667615 26983 leader_election.cc:302] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cff7af6dbc94e638325d0478336439a, 4d8dac4393df49b5a7a17d5d844bd538; no voters:
I20260806 15:59:06.667757 27439 raft_consensus.cc:2781] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:06.667845 27439 raft_consensus.cc:686] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [term 1 LEADER]: Becoming Leader. State: Replica: 2cff7af6dbc94e638325d0478336439a, State: Running, Role: LEADER
I20260806 15:59:06.667989 27439 consensus_queue.cc:237] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.669669 26890 catalog_manager.cc:5495] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cff7af6dbc94e638325d0478336439a (127.25.31.129). New cstate: current_term: 1 leader_uuid: "2cff7af6dbc94e638325d0478336439a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } health_report { overall_health: UNKNOWN } } }
Aug 06 15:59:06 dist-test-slave-tm1g krb5kdc[26844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031945, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:06.713416 27453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.704165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:43381 (local address 127.0.0.1:51402)
0806 15:59:06.704559 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:06.704579 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:06.704724 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:06.704992 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:06.704995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:06.705519 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:06.705642 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.705650 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.706500 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.706503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.707327 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.707339 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.707826 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.707830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.707923 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.708551 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:06.708566 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:06.710170 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:06.711827 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:06.711838 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:06.711854 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:06.712078 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:06.712358 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:06.712361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:06.712363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:06.712449 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:06.712817 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:06.712824 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:06.712943 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 15:59:06.713058 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 15:59:06.713213 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":194,"threads_started":1}
I20260806 15:59:06.713416 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.704186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51402 (local address 127.25.31.190:43381)
0806 15:59:06.704339 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:06.704344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:06.704675 (+ 331us) server_negotiation.cc:407] Connection header received
0806 15:59:06.704818 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:06.704821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:06.704865 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:06.704934 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:06.705773 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.706298 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.707484 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.707697 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.707783 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.710336 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:06.710349 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:06.710352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:06.710378 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:06.711716 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:06.712179 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:06.712184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:06.712186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:06.712260 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:06.712535 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:06.712540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:06.712541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:06.712710 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:06.712816 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:06.713106 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 15:59:06.713269 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":52}
I20260806 15:59:06.717509 26890 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:51402:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 15:59:06.718257 26890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:06.723531 27032 tablet_service.cc:1467] Processing CreateTablet for tablet a39acb432c2645b792a8ba40fa656956 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3b8d6f01704c4ff393df2f75db6c698e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:59:06.723946 27032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a39acb432c2645b792a8ba40fa656956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:06.723816 27323 tablet_service.cc:1467] Processing CreateTablet for tablet a39acb432c2645b792a8ba40fa656956 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3b8d6f01704c4ff393df2f75db6c698e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:59:06.723827 27178 tablet_service.cc:1467] Processing CreateTablet for tablet a39acb432c2645b792a8ba40fa656956 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3b8d6f01704c4ff393df2f75db6c698e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:59:06.724205 27323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a39acb432c2645b792a8ba40fa656956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:06.724203 27178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a39acb432c2645b792a8ba40fa656956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:06.727156 27415 tablet_bootstrap.cc:492] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538: Bootstrap starting.
I20260806 15:59:06.727528 27414 tablet_bootstrap.cc:492] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5: Bootstrap starting.
I20260806 15:59:06.727952 27415 tablet_bootstrap.cc:654] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:06.728519 27414 tablet_bootstrap.cc:654] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:06.729952 27416 tablet_bootstrap.cc:492] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a: Bootstrap starting.
I20260806 15:59:06.729967 27414 tablet_bootstrap.cc:492] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5: No bootstrap required, opened a new log
I20260806 15:59:06.730047 27414 ts_tablet_manager.cc:1397] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 15:59:06.730691 27414 raft_consensus.cc:348] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.730832 27414 raft_consensus.cc:374] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:06.730875 27414 raft_consensus.cc:729] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9f40b680bf440b7a7b849af709ee4f5, State: Initialized, Role: FOLLOWER
I20260806 15:59:06.730888 27415 tablet_bootstrap.cc:492] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538: No bootstrap required, opened a new log
I20260806 15:59:06.730978 27415 ts_tablet_manager.cc:1397] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 15:59:06.731029 27416 tablet_bootstrap.cc:654] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:06.731165 27414 consensus_queue.cc:260] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.731463 27415 raft_consensus.cc:348] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.731587 27415 raft_consensus.cc:374] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:06.731626 27415 raft_consensus.cc:729] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8dac4393df49b5a7a17d5d844bd538, State: Initialized, Role: FOLLOWER
I20260806 15:59:06.731770 27415 consensus_queue.cc:260] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.731757 27420 tablet_replica.cc:439] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } }
I20260806 15:59:06.731928 27420 tablet_replica.cc:442] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:06.732029 27415 ts_tablet_manager.cc:1428] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 15:59:06.732132 27421 tablet_replica.cc:439] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } }
I20260806 15:59:06.732270 27421 tablet_replica.cc:442] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:06.733096 27414 ts_tablet_manager.cc:1428] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 15:59:06.734699 27416 tablet_bootstrap.cc:492] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a: No bootstrap required, opened a new log
I20260806 15:59:06.734790 27416 ts_tablet_manager.cc:1397] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260806 15:59:06.735155 27416 raft_consensus.cc:348] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.735251 27416 raft_consensus.cc:374] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:06.735273 27416 raft_consensus.cc:729] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cff7af6dbc94e638325d0478336439a, State: Initialized, Role: FOLLOWER
I20260806 15:59:06.735404 27416 consensus_queue.cc:260] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.735694 27416 ts_tablet_manager.cc:1428] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 15:59:06.735639 27439 tablet_replica.cc:439] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } }
I20260806 15:59:06.735764 27439 tablet_replica.cc:442] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 15:59:06.840034 27102 tablet.cc:2369] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:59:06.962658 27421 raft_consensus.cc:482] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:06.962821 27421 raft_consensus.cc:504] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.963946 27421 leader_election.cc:288] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cff7af6dbc94e638325d0478336439a (127.25.31.129:41073), f9f40b680bf440b7a7b849af709ee4f5 (127.25.31.131:35839)
I20260806 15:59:06.968446 27409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.964368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:60253 (local address 127.25.31.129:41073)
0806 15:59:06.964504 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:06.964510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:06.964774 (+ 264us) server_negotiation.cc:407] Connection header received
0806 15:59:06.964929 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:06.964931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:06.965008 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:06.965086 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:06.965394 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.965890 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.967334 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.967827 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.967949 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.968014 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:06.968245 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 15:59:06.968325 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 15:59:06.968472 27412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.964326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:32917 (local address 127.25.31.131:35839)
0806 15:59:06.964468 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:06.964473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:06.964700 (+ 227us) server_negotiation.cc:407] Connection header received
0806 15:59:06.964860 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:06.964865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:06.964930 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:06.964989 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:06.965360 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.965880 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.967400 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.967852 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.967986 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.968041 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:06.968281 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 15:59:06.968364 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 15:59:06.968468 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.964215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:41073 (local address 127.25.31.130:60253)
0806 15:59:06.964632 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:06.964644 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:06.964805 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:06.965119 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:06.965122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:06.965145 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:06.965241 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.965246 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.966010 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.966013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.967206 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.967220 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.967951 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.967954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.968071 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.968132 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 15:59:06.968193 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 15:59:06.968283 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":92,"thread_start_us":124,"threads_started":1}
I20260806 15:59:06.968475 27458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:06.964209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:35839 (local address 127.25.31.130:32917)
0806 15:59:06.964593 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:06.964605 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:06.964710 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:06.965031 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:06.965037 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:06.965059 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:06.965203 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.965210 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.966007 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.966011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.967241 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:06.967251 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:06.968001 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:06.968004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:06.968131 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:06.968148 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 15:59:06.968243 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 15:59:06.968314 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":64,"thread_start_us":102,"threads_started":1}
I20260806 15:59:06.968953 27053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a39acb432c2645b792a8ba40fa656956" candidate_uuid: "4d8dac4393df49b5a7a17d5d844bd538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cff7af6dbc94e638325d0478336439a" is_pre_election: true
I20260806 15:59:06.969132 27342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a39acb432c2645b792a8ba40fa656956" candidate_uuid: "4d8dac4393df49b5a7a17d5d844bd538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9f40b680bf440b7a7b849af709ee4f5" is_pre_election: true
I20260806 15:59:06.969229 27053 raft_consensus.cc:2454] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d8dac4393df49b5a7a17d5d844bd538 in term 0.
I20260806 15:59:06.969300 27342 raft_consensus.cc:2454] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d8dac4393df49b5a7a17d5d844bd538 in term 0.
I20260806 15:59:06.969645 27130 leader_election.cc:302] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d8dac4393df49b5a7a17d5d844bd538, f9f40b680bf440b7a7b849af709ee4f5; no voters:
I20260806 15:59:06.969918 27421 raft_consensus.cc:2781] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:06.969987 27421 raft_consensus.cc:482] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:06.970026 27421 raft_consensus.cc:3037] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:06.971148 27421 raft_consensus.cc:504] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.971484 27421 leader_election.cc:288] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [CANDIDATE]: Term 1 election: Requested vote from peers 2cff7af6dbc94e638325d0478336439a (127.25.31.129:41073), f9f40b680bf440b7a7b849af709ee4f5 (127.25.31.131:35839)
I20260806 15:59:06.971801 27053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a39acb432c2645b792a8ba40fa656956" candidate_uuid: "4d8dac4393df49b5a7a17d5d844bd538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cff7af6dbc94e638325d0478336439a"
I20260806 15:59:06.971920 27053 raft_consensus.cc:3037] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:06.971905 27342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a39acb432c2645b792a8ba40fa656956" candidate_uuid: "4d8dac4393df49b5a7a17d5d844bd538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9f40b680bf440b7a7b849af709ee4f5"
I20260806 15:59:06.972046 27342 raft_consensus.cc:3037] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:06.972901 27053 raft_consensus.cc:2454] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d8dac4393df49b5a7a17d5d844bd538 in term 1.
I20260806 15:59:06.973004 27342 raft_consensus.cc:2454] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d8dac4393df49b5a7a17d5d844bd538 in term 1.
I20260806 15:59:06.973178 27127 leader_election.cc:302] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cff7af6dbc94e638325d0478336439a, 4d8dac4393df49b5a7a17d5d844bd538; no voters:
I20260806 15:59:06.973397 27421 raft_consensus.cc:2781] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:06.973613 27421 raft_consensus.cc:686] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [term 1 LEADER]: Becoming Leader. State: Replica: 4d8dac4393df49b5a7a17d5d844bd538, State: Running, Role: LEADER
I20260806 15:59:06.973860 27421 consensus_queue.cc:237] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } }
I20260806 15:59:06.975265 27459 tablet_replica.cc:439] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538: TxnStatusTablet state changed. Reason: New leader 4d8dac4393df49b5a7a17d5d844bd538. Latest consensus state: current_term: 1 leader_uuid: "4d8dac4393df49b5a7a17d5d844bd538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } }
I20260806 15:59:06.975399 27459 tablet_replica.cc:442] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:59:06.975724 27461 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 15:59:06.975939 27461 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 15:59:06.976600 26892 catalog_manager.cc:5495] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d8dac4393df49b5a7a17d5d844bd538 (127.25.31.130). New cstate: current_term: 1 leader_uuid: "4d8dac4393df49b5a7a17d5d844bd538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260806 15:59:07.036046 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:07.031443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40434 (local address 127.25.31.130:39589)
0806 15:59:07.031683 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:07.031686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:07.031700 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:59:07.031775 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:07.031777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:07.031832 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:07.031928 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:07.032190 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.032759 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.033967 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.034793 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.034891 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:07.035226 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:07.035658 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:07.035748 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:07.035858 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 15:59:07.035909 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":81,"threads_started":1}
I20260806 15:59:07.036094 27453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:07.031268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:39589 (local address 127.0.0.1:40434)
0806 15:59:07.031511 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:07.031527 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:07.031626 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:07.031973 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:07.031976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:07.031984 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:59:07.032102 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:07.032109 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.032863 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.032867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:07.033826 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:07.033837 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.034910 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.034914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:07.035025 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:07.035079 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:59:07.035755 (+ 676us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:59:07.035770 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 15:59:07.035849 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 15:59:07.035938 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260806 15:59:07.039403 27342 raft_consensus.cc:1261] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5 [term 1 FOLLOWER]: Refusing update from remote peer 4d8dac4393df49b5a7a17d5d844bd538: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:07.039589 27053 raft_consensus.cc:1261] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a [term 1 FOLLOWER]: Refusing update from remote peer 4d8dac4393df49b5a7a17d5d844bd538: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:07.040005 27421 consensus_queue.cc:1035] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:07.040519 27421 consensus_queue.cc:1035] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:07.041405 27420 tablet_replica.cc:439] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5: TxnStatusTablet state changed. Reason: New leader 4d8dac4393df49b5a7a17d5d844bd538. Latest consensus state: current_term: 1 leader_uuid: "4d8dac4393df49b5a7a17d5d844bd538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } }
I20260806 15:59:07.041527 27420 tablet_replica.cc:442] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:07.042838 27439 tablet_replica.cc:439] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a: TxnStatusTablet state changed. Reason: New leader 4d8dac4393df49b5a7a17d5d844bd538. Latest consensus state: current_term: 1 leader_uuid: "4d8dac4393df49b5a7a17d5d844bd538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } }
I20260806 15:59:07.042989 27439 tablet_replica.cc:442] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:07.044266 27421 tablet_replica.cc:439] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4d8dac4393df49b5a7a17d5d844bd538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } }
I20260806 15:59:07.044378 27421 tablet_replica.cc:442] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:59:07.045583 27420 tablet_replica.cc:439] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4d8dac4393df49b5a7a17d5d844bd538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } }
I20260806 15:59:07.045689 27420 tablet_replica.cc:442] T a39acb432c2645b792a8ba40fa656956 P f9f40b680bf440b7a7b849af709ee4f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:07.047953 27439 consensus_queue.cc:1035] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:07.049116 27459 tablet_replica.cc:439] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4d8dac4393df49b5a7a17d5d844bd538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } }
I20260806 15:59:07.049225 27459 tablet_replica.cc:442] T a39acb432c2645b792a8ba40fa656956 P 4d8dac4393df49b5a7a17d5d844bd538: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:59:07.052518 27439 consensus_queue.cc:1035] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:07.054908 27429 mvcc.cc:205] Tried to move back new op lower bound from 7315586855068094464 to 7315586854807453696. Current Snapshot: MvccSnapshot[applied={T|T < 7315586855068094464}]
I20260806 15:59:07.055083 27439 tablet_replica.cc:439] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4d8dac4393df49b5a7a17d5d844bd538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cff7af6dbc94e638325d0478336439a" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41073 } } peers { permanent_uuid: "4d8dac4393df49b5a7a17d5d844bd538" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39589 } } peers { permanent_uuid: "f9f40b680bf440b7a7b849af709ee4f5" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35839 } } }
I20260806 15:59:07.055176 27439 tablet_replica.cc:442] T a39acb432c2645b792a8ba40fa656956 P 2cff7af6dbc94e638325d0478336439a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:07.067034 27472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:07.062666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:39589 (local address 127.25.31.129:39311)
0806 15:59:07.063004 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:07.063021 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:07.063116 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:07.063366 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:07.063368 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:07.063377 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:59:07.063491 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:07.063497 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.064576 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.064579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:07.065629 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:07.065642 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.066266 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.066269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:07.066388 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:07.066440 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:59:07.066742 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:59:07.066747 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:59:07.066799 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 15:59:07.066880 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":107,"threads_started":1}
I20260806 15:59:07.067384 27464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:07.063020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:39311 (local address 127.25.31.130:39589)
0806 15:59:07.063133 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:07.063138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:07.063179 (+ 41us) server_negotiation.cc:407] Connection header received
0806 15:59:07.063222 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:07.063225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:07.063271 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:07.063342 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:07.063620 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.064440 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.065770 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.066109 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.066192 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:07.066579 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:07.066657 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:07.066712 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:07.067209 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 15:59:07.067263 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 15:59:07.095355 27409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:07.091585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:49011 (local address 127.25.31.129:41073)
0806 15:59:07.091713 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:07.091718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:07.091989 (+ 271us) server_negotiation.cc:407] Connection header received
0806 15:59:07.092088 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:07.092090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:07.092131 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:07.092191 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:07.092477 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.093089 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.094267 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.094579 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.094661 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:07.094969 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:07.095034 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:07.095089 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:07.095203 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 15:59:07.095249 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 15:59:07.095408 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:07.091540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:41073 (local address 127.25.31.130:49011)
0806 15:59:07.091882 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:07.091897 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:07.092012 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:07.092223 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:07.092225 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:07.092233 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:59:07.092368 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:07.092377 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.093183 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.093186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:07.094076 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:07.094087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.094688 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.094691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:07.094787 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:07.094837 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:59:07.095116 (+ 279us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:59:07.095123 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 15:59:07.095191 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 15:59:07.095267 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":118,"threads_started":1}
I20260806 15:59:07.120466 26892 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:07.120659 26892 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:07.122797 26892 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 973d336e1782420a86a1c6731f9f2e56: Sending DeleteTablet for 3 replicas of tablet e69a7828350442cb8b77a68370165c90
I20260806 15:59:07.123355 27032 tablet_service.cc:1514] Processing DeleteTablet for tablet e69a7828350442cb8b77a68370165c90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:07 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:59662
I20260806 15:59:07.123513 27178 tablet_service.cc:1514] Processing DeleteTablet for tablet e69a7828350442cb8b77a68370165c90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:07 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:40432
I20260806 15:59:07.123580 27443 tablet_replica.cc:330] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a: stopping tablet replica
I20260806 15:59:07.123703 27443 raft_consensus.cc:2229] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:59:07.123926 27443 raft_consensus.cc:2258] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:07.124013 27477 tablet_replica.cc:330] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538: stopping tablet replica
I20260806 15:59:07.124019 27323 tablet_service.cc:1514] Processing DeleteTablet for tablet e69a7828350442cb8b77a68370165c90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:07 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:47234
I20260806 15:59:07.124132 27477 raft_consensus.cc:2229] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:07.124236 27477 raft_consensus.cc:2258] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:07.124699 27443 ts_tablet_manager.cc:1905] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:07.126151 27477 ts_tablet_manager.cc:1905] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:07.127943 27443 ts_tablet_manager.cc:1918] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:59:07.128021 27443 log.cc:1196] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-0/wal/wals/e69a7828350442cb8b77a68370165c90
I20260806 15:59:07.128271 27443 ts_tablet_manager.cc:1939] T e69a7828350442cb8b77a68370165c90 P 2cff7af6dbc94e638325d0478336439a: Deleting consensus metadata
I20260806 15:59:07.129120 26874 catalog_manager.cc:4841] TS 2cff7af6dbc94e638325d0478336439a (127.25.31.129:41073): tablet e69a7828350442cb8b77a68370165c90 (table test-table [id=a9e93d72d070470c818bcf641638537e]) successfully deleted
I20260806 15:59:07.129323 27478 tablet_replica.cc:330] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5: stopping tablet replica
I20260806 15:59:07.129387 27477 ts_tablet_manager.cc:1918] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:59:07.129464 27478 raft_consensus.cc:2229] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:07.129484 27477 log.cc:1196] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/e69a7828350442cb8b77a68370165c90
I20260806 15:59:07.129591 27478 raft_consensus.cc:2258] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:07.129774 27477 ts_tablet_manager.cc:1939] T e69a7828350442cb8b77a68370165c90 P 4d8dac4393df49b5a7a17d5d844bd538: Deleting consensus metadata
I20260806 15:59:07.130283 27478 ts_tablet_manager.cc:1905] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:07.130460 26875 catalog_manager.cc:4841] TS 4d8dac4393df49b5a7a17d5d844bd538 (127.25.31.130:39589): tablet e69a7828350442cb8b77a68370165c90 (table test-table [id=a9e93d72d070470c818bcf641638537e]) successfully deleted
Aug 06 15:59:07 dist-test-slave-tm1g krb5kdc[26844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031946, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.31.129@KRBTEST.COM
I20260806 15:59:07.133635 27478 ts_tablet_manager.cc:1918] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:59:07.133713 27478 log.cc:1196] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/e69a7828350442cb8b77a68370165c90
I20260806 15:59:07.133984 27478 ts_tablet_manager.cc:1939] T e69a7828350442cb8b77a68370165c90 P f9f40b680bf440b7a7b849af709ee4f5: Deleting consensus metadata
I20260806 15:59:07.134832 26877 catalog_manager.cc:4841] TS f9f40b680bf440b7a7b849af709ee4f5 (127.25.31.131:35839): tablet e69a7828350442cb8b77a68370165c90 (table test-table [id=a9e93d72d070470c818bcf641638537e]) successfully deleted
I20260806 15:59:07.138341 27409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:07.125731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59684 (local address 127.25.31.129:41073)
0806 15:59:07.125854 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:07.125859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:07.126109 (+ 250us) server_negotiation.cc:407] Connection header received
0806 15:59:07.126412 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:07.126418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:07.126481 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:07.126538 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:07.127612 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.128451 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.130083 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.130428 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.130508 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:07.134466 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:07.134480 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:07.134496 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:07.134525 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:07.136448 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:07.136893 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:07.136907 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:07.136911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:07.136999 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:07.137304 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:07.137308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:07.137310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:07.137634 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:07.137864 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:07.138022 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 15:59:07.138151 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 15:59:07.138911 27095 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59684
I20260806 15:59:07.148877 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:07.140848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51412 (local address 127.25.31.190:43381)
0806 15:59:07.140978 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:07.140983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:07.141086 (+ 103us) server_negotiation.cc:407] Connection header received
0806 15:59:07.141179 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:07.141183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:07.141235 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:07.141338 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:07.141916 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.142782 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.143899 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:07.144276 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:07.144390 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:07.145655 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:07.145659 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:07.145662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:07.145688 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:07.147265 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:07.147697 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:07.147701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:07.147703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:07.147800 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:07.148069 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:07.148072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:07.148074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:07.148279 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:07.148402 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:07.148598 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 15:59:07.148716 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 15:59:07.149250 26892 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51412
I20260806 15:59:07.150417 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 26965
I20260806 15:59:07.160853 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 27110
I20260806 15:59:07.168527 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 27257
I20260806 15:59:07.174441 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 26858
2026-08-06T15:59:07Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2973 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:59:07 dist-test-slave-tm1g krb5kdc[27485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:59:07 dist-test-slave-tm1g krb5kdc[27485](info): set up 2 sockets
Aug 06 15:59:07 dist-test-slave-tm1g krb5kdc[27485](info): commencing operation
krb5kdc: starting...
W20260806 15:59:08.230767 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 15:59:08 dist-test-slave-tm1g krb5kdc[27485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:59:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 15:59:08.349295 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42679
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:43511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:42679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:08.458618 27499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:08.458911 27499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:08.458986 27499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:08.462889 27499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:08.462971 27499 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 15:59:08.463004 27499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:08.463064 27499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:08.463120 27499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:08.463158 27499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:08.468364 27499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:43511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:42679
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:08.469393 27499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:08.470443 27499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:08.478559 27505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:08.478576 27507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:08.478631 27499 server_base.cc:1034] running on GCE node
W20260806 15:59:08.478591 27504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:08.479276 27499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:08.480309 27499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:08.481500 27499 hybrid_clock.cc:648] HybridClock initialized: now 1786031948481482 us; error 30 us; skew 500 ppm
Aug 06 15:59:08 dist-test-slave-tm1g krb5kdc[27485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031948, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:08.487564 27499 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 15:59:08.488461 27499 webserver.cc:460] Webserver started at http://127.25.31.190:44619/ using document root <none> and password file <none>
I20260806 15:59:08.489006 27499 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:08.489097 27499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:08.489331 27499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:08.491348 27499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "3c0e5acbab6045b8b6ec81c1bd02416e"
format_stamp: "Formatted at 2026-08-06 15:59:08 on dist-test-slave-tm1g"
server_key: "a298eb57adff3029fa7e3ca947cb691e"
server_key_iv: "6bdadb6cb7dc26beb361e0008ffc178f"
server_key_version: "encryptionkey@0"
I20260806 15:59:08.491920 27499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "3c0e5acbab6045b8b6ec81c1bd02416e"
format_stamp: "Formatted at 2026-08-06 15:59:08 on dist-test-slave-tm1g"
server_key: "a298eb57adff3029fa7e3ca947cb691e"
server_key_iv: "6bdadb6cb7dc26beb361e0008ffc178f"
server_key_version: "encryptionkey@0"
I20260806 15:59:08.495572 27499 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 15:59:08.497931 27514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:08.499249 27499 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:08.499395 27499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "3c0e5acbab6045b8b6ec81c1bd02416e"
format_stamp: "Formatted at 2026-08-06 15:59:08 on dist-test-slave-tm1g"
server_key: "a298eb57adff3029fa7e3ca947cb691e"
server_key_iv: "6bdadb6cb7dc26beb361e0008ffc178f"
server_key_version: "encryptionkey@0"
I20260806 15:59:08.499521 27499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:08.517031 27499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:08.519923 27499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:08.520124 27499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:08.528146 27499 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:42679
I20260806 15:59:08.528170 27579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:42679 every 8 connection(s)
I20260806 15:59:08.529066 27499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:08.531737 27580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:08.535229 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 27499
I20260806 15:59:08.535320 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:08.535571 25726 external_mini_cluster.cc:1428] Setting key 88b2c17d87d51a03d05416836de14334
I20260806 15:59:08.537428 27580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e: Bootstrap starting.
I20260806 15:59:08.540097 27580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:08.540968 27580 log.cc:824] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:08.543048 27580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e: No bootstrap required, opened a new log
Aug 06 15:59:08 dist-test-slave-tm1g krb5kdc[27485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:08.547101 27580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0e5acbab6045b8b6ec81c1bd02416e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42679 } }
I20260806 15:59:08.547374 27580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:08.547423 27580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c0e5acbab6045b8b6ec81c1bd02416e, State: Initialized, Role: FOLLOWER
I20260806 15:59:08.547914 27580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0e5acbab6045b8b6ec81c1bd02416e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42679 } }
I20260806 15:59:08.548100 27580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:08.548187 27580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:08.548285 27580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:08.549384 27580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0e5acbab6045b8b6ec81c1bd02416e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42679 } }
I20260806 15:59:08.549739 27580 leader_election.cc:302] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c0e5acbab6045b8b6ec81c1bd02416e; no voters:
I20260806 15:59:08.550105 27580 leader_election.cc:288] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:08.550241 27585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:08.550509 27585 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [term 1 LEADER]: Becoming Leader. State: Replica: 3c0e5acbab6045b8b6ec81c1bd02416e, State: Running, Role: LEADER
I20260806 15:59:08.550797 27585 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0e5acbab6045b8b6ec81c1bd02416e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42679 } }
I20260806 15:59:08.551165 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:08.536942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42854 (local address 127.25.31.190:42679)
0806 15:59:08.537342 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:08.537360 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:08.537402 (+ 42us) server_negotiation.cc:407] Connection header received
0806 15:59:08.538192 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:08.538214 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:08.538662 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:08.539011 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:08.539656 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:08.541175 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:08.542212 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:08.542918 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:08.543094 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:08.545778 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:08.545784 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:08.545801 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:08.545832 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:08.547932 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:08.548507 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:08.548513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:08.548519 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:08.548651 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:08.548912 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:08.548915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:08.548917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:08.549450 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:08.549626 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:08.549930 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 15:59:08.550228 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":113,"threads_started":1}
I20260806 15:59:08.551326 27580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:08.552404 27586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c0e5acbab6045b8b6ec81c1bd02416e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0e5acbab6045b8b6ec81c1bd02416e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42679 } } }
I20260806 15:59:08.552511 27586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:08.553002 27587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c0e5acbab6045b8b6ec81c1bd02416e. Latest consensus state: current_term: 1 leader_uuid: "3c0e5acbab6045b8b6ec81c1bd02416e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0e5acbab6045b8b6ec81c1bd02416e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42679 } } }
I20260806 15:59:08.553143 27587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e [sys.catalog]: This master's current role is: LEADER
W20260806 15:59:08.556072 27600 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:59:08.556147 27600 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:59:08.556205 27594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:08.557159 27594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:08.563122 27594 catalog_manager.cc:1347] Generated new cluster ID: 2125fdd839344af997fb61fc7f399312
I20260806 15:59:08.563194 27594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:08.584645 27594 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:08.585594 27594 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:08.591084 27594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e: Generated new TSK 0
I20260806 15:59:08.591741 27594 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 15:59:08.633195 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42679
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:43511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:08.753197 27606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:08.753435 27606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:08.753511 27606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:08.757431 27606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:08.757485 27606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:08.757524 27606 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 15:59:08.757537 27606 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:59:08.757614 27606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:08.762688 27606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:43511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.31.190:42679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:08.763646 27606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:08.764621 27606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:08.774278 27615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:08.774377 27606 server_base.cc:1034] running on GCE node
W20260806 15:59:08.774282 27613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:08.774504 27612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:08.774915 27606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:08.775491 27606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:08.776690 27606 hybrid_clock.cc:648] HybridClock initialized: now 1786031948776655 us; error 44 us; skew 500 ppm
Aug 06 15:59:08 dist-test-slave-tm1g krb5kdc[27485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031948, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:08.779506 27606 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 15:59:08.780329 27606 webserver.cc:460] Webserver started at http://127.25.31.129:36791/ using document root <none> and password file <none>
I20260806 15:59:08.780882 27606 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:08.780968 27606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:08.781186 27606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:08.783231 27606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "9f76b7d35635404b95797b18318d42c0"
format_stamp: "Formatted at 2026-08-06 15:59:08 on dist-test-slave-tm1g"
server_key: "31862582fbd1478c7e45d923aa4d513a"
server_key_iv: "ff0025b3a9c41859cbce051553e2e4df"
server_key_version: "encryptionkey@0"
I20260806 15:59:08.783797 27606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "9f76b7d35635404b95797b18318d42c0"
format_stamp: "Formatted at 2026-08-06 15:59:08 on dist-test-slave-tm1g"
server_key: "31862582fbd1478c7e45d923aa4d513a"
server_key_iv: "ff0025b3a9c41859cbce051553e2e4df"
server_key_version: "encryptionkey@0"
I20260806 15:59:08.787441 27606 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 15:59:08.789786 27622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:08.791062 27606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 15:59:08.791209 27606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "9f76b7d35635404b95797b18318d42c0"
format_stamp: "Formatted at 2026-08-06 15:59:08 on dist-test-slave-tm1g"
server_key: "31862582fbd1478c7e45d923aa4d513a"
server_key_iv: "ff0025b3a9c41859cbce051553e2e4df"
server_key_version: "encryptionkey@0"
I20260806 15:59:08.791323 27606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:08.832332 27606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:08.835516 27606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:08.835741 27606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:08.837211 27606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:08.837285 27606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:08.837357 27606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:08.837400 27606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 15:59:08 dist-test-slave-tm1g krb5kdc[27485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031948, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:08.848181 27606 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:39385
I20260806 15:59:08.848393 27740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:39385 every 8 connection(s)
I20260806 15:59:08.849115 27606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:08.850211 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 27606
I20260806 15:59:08.850369 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 15:59:08.850642 25726 external_mini_cluster.cc:1428] Setting key 1bac0fa8d1fb6da6546ff30980677b10
I20260806 15:59:08.851785 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:08.838166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:39031 (local address 127.25.31.190:42679)
0806 15:59:08.838333 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:08.838339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:08.839458 (+ 1119us) server_negotiation.cc:407] Connection header received
0806 15:59:08.840724 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:08.840729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:08.840795 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:08.840898 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:08.842643 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:08.843275 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:08.844435 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:08.844682 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:08.844787 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:08.847672 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:08.847675 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:08.847677 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:08.847701 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:08.849445 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:08.850074 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:08.850077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:08.850079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:08.850168 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:08.850576 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:08.850580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:08.850582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:08.850827 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:08.850937 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:08.851469 (+ 532us) server_negotiation.cc:299] Negotiation successful
0806 15:59:08.851613 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 15:59:08.852648 27631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:08.838740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42679 (local address 127.25.31.129:39031)
0806 15:59:08.839256 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:08.839301 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:08.840479 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:08.841251 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:08.841258 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:08.841743 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:08.842470 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:08.842509 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:08.843415 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:08.843419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:08.844302 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:08.844311 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:08.844850 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:08.844856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:08.845026 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:08.845662 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:08.845678 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:08.847485 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:08.849585 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:08.849590 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:08.849604 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:08.849938 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:08.850269 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:08.850272 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:08.850274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:08.850485 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:08.850932 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:08.850939 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:08.851224 (+ 285us) client_negotiation.cc:769] Sending connection context
0806 15:59:08.851410 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 15:59:08.851723 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":137,"threads_started":1}
I20260806 15:59:08.853926 27741 heartbeater.cc:347] Connected to a master server at 127.25.31.190:42679
I20260806 15:59:08.854065 27741 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:08.854658 27741 heartbeater.cc:511] Master 127.25.31.190:42679 requested a full tablet report, sending...
I20260806 15:59:08.856384 27532 ts_manager.cc:194] Registered new tserver with Master: 9f76b7d35635404b95797b18318d42c0 (127.25.31.129:39385)
I20260806 15:59:08.857503 27532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:39031
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
I20260806 15:59:08.865437 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:08.856091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:36273 (local address 127.25.31.190:42679)
0806 15:59:08.858044 (+ 1953us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:08.858047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:08.858082 (+ 35us) server_negotiation.cc:407] Connection header received
0806 15:59:08.858116 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:08.858119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:08.858157 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:08.858242 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:08.858885 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:08.859612 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:08.860882 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:08.861088 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:08.861200 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:08.862324 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:08.862329 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:08.862331 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:08.862355 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:08.863786 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:08.864261 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:08.864266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:08.864267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:08.864346 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:08.864627 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:08.864632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:08.864633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:08.864801 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:08.864916 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:08.865169 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 15:59:08.865276 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1898}
I20260806 15:59:08.865450 27749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:08.856035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42679 (local address 127.25.31.129:36273)
0806 15:59:08.856299 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:08.856313 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:08.856398 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:08.858291 (+ 1893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:08.858296 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:08.858575 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:08.858740 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:08.858750 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:08.859763 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:08.859769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:08.860755 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:08.860769 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:08.861207 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:08.861210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:08.861316 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:08.861819 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:08.861833 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:08.862169 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:08.863897 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:08.863902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:08.863905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:08.864163 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:08.864438 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:08.864442 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:08.864444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:08.864537 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:08.864908 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:08.864911 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:08.865008 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 15:59:08.865115 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 15:59:08.865255 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":101,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 15:59:08.879321 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42679
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:43511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:08.989645 27751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:08.989930 27751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:08.990018 27751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:08.994047 27751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:08.994141 27751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:08.994216 27751 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 15:59:08.994262 27751 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:59:08.994356 27751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:08.999733 27751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:43511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.31.190:42679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:09.000792 27751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:09.001821 27751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:59:09.011724 27751 server_base.cc:1034] running on GCE node
W20260806 15:59:09.011703 27757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:09.011917 27756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:09.011703 27759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:09.012410 27751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:09.013090 27751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:09.014281 27751 hybrid_clock.cc:648] HybridClock initialized: now 1786031949014254 us; error 50 us; skew 500 ppm
Aug 06 15:59:09 dist-test-slave-tm1g krb5kdc[27485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031949, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:09.017191 27751 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 15:59:09.018121 27751 webserver.cc:460] Webserver started at http://127.25.31.130:36257/ using document root <none> and password file <none>
I20260806 15:59:09.018739 27751 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:09.018831 27751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:09.019068 27751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:09.021107 27751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "d872c956af834b7c8d8e1237af06f06c"
format_stamp: "Formatted at 2026-08-06 15:59:09 on dist-test-slave-tm1g"
server_key: "ef97c374376198c1c148689f2e7a32e6"
server_key_iv: "32838b3ce31313a6889eb5ad77cfce8b"
server_key_version: "encryptionkey@0"
I20260806 15:59:09.021684 27751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "d872c956af834b7c8d8e1237af06f06c"
format_stamp: "Formatted at 2026-08-06 15:59:09 on dist-test-slave-tm1g"
server_key: "ef97c374376198c1c148689f2e7a32e6"
server_key_iv: "32838b3ce31313a6889eb5ad77cfce8b"
server_key_version: "encryptionkey@0"
I20260806 15:59:09.025480 27751 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 15:59:09.027982 27766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:09.029305 27751 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 15:59:09.029458 27751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "d872c956af834b7c8d8e1237af06f06c"
format_stamp: "Formatted at 2026-08-06 15:59:09 on dist-test-slave-tm1g"
server_key: "ef97c374376198c1c148689f2e7a32e6"
server_key_iv: "32838b3ce31313a6889eb5ad77cfce8b"
server_key_version: "encryptionkey@0"
I20260806 15:59:09.029573 27751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:09.048642 27751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:09.052213 27751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:09.052444 27751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:09.054476 27751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:09.054561 27751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:09.054627 27751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:09.054656 27751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 15:59:09 dist-test-slave-tm1g krb5kdc[27485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031949, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:09.065662 27751 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:39883
I20260806 15:59:09.065718 27888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:39883 every 8 connection(s)
I20260806 15:59:09.066685 27751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:09.068300 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.054979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:53411 (local address 127.25.31.190:42679)
0806 15:59:09.055140 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.055148 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.056025 (+ 877us) server_negotiation.cc:407] Connection header received
0806 15:59:09.056966 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.056973 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.057036 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.057137 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:09.058412 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.059110 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.061039 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.061394 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.061533 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.064434 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:09.064437 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:09.064439 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:09.064463 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:09.066045 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:09.066647 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:09.066656 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:09.066657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:09.066770 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:09.067134 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:09.067140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:09.067143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:09.067383 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:09.067496 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.068075 (+ 579us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.068182 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 15:59:09.069006 27775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.055427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42679 (local address 127.25.31.130:53411)
0806 15:59:09.055855 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:09.055912 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:09.056765 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:09.057349 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:09.057358 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:09.057774 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:09.058225 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.058238 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.059273 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.059278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.060902 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.060915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.061605 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.061613 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.061859 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.062586 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:09.062605 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:09.064256 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:09.066165 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:09.066173 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:09.066187 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:09.066536 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:09.066877 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:09.066880 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:09.066882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:09.067029 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:09.067550 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:09.067555 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:09.067804 (+ 249us) client_negotiation.cc:769] Sending connection context
0806 15:59:09.067951 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 15:59:09.068188 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":120,"threads_started":1}
I20260806 15:59:09.070247 27889 heartbeater.cc:347] Connected to a master server at 127.25.31.190:42679
I20260806 15:59:09.070456 27889 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:09.070971 27889 heartbeater.cc:511] Master 127.25.31.190:42679 requested a full tablet report, sending...
I20260806 15:59:09.072002 27532 ts_manager.cc:194] Registered new tserver with Master: d872c956af834b7c8d8e1237af06f06c (127.25.31.130:39883)
I20260806 15:59:09.072598 27532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:53411
I20260806 15:59:09.075790 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 27751
I20260806 15:59:09.075908 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 15:59:09.076225 25726 external_mini_cluster.cc:1428] Setting key c5bde95e1d4bb2ebeb6242b5045018cc
I20260806 15:59:09.080541 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.073103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:50769 (local address 127.25.31.190:42679)
0806 15:59:09.073224 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.073229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.073409 (+ 180us) server_negotiation.cc:407] Connection header received
0806 15:59:09.073525 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.073529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.073573 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.073655 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:09.074097 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.074668 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.075497 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.075848 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.075960 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.077199 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:09.077203 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:09.077205 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:09.077231 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:09.078670 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:09.079237 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:09.079241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:09.079242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:09.079333 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:09.079659 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:09.079664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:09.079665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:09.079882 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:09.079993 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.080263 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.080396 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 15:59:09.080551 27895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.073041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42679 (local address 127.25.31.130:50769)
0806 15:59:09.073325 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:09.073337 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:09.073450 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:09.073671 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:09.073674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:09.073885 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:09.073981 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.073986 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.074791 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.074795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.075369 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.075377 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.076090 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.076093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.076189 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.076637 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:09.076651 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:09.077068 (+ 417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:09.078775 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:09.078781 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:09.078786 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:09.079072 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:09.079458 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:09.079461 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:09.079463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:09.079561 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:09.079994 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:09.079997 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:09.080096 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 15:59:09.080204 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 15:59:09.080355 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 15:59:09.105412 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42679
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:43511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:09.216217 27898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:09.216495 27898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:09.216578 27898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:09.220558 27898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:09.220642 27898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:09.220714 27898 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 15:59:09.220758 27898 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:59:09.220821 27898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:09.225908 27898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:43511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.31.190:42679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:09.226963 27898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:09.227941 27898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:09.237344 27907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:09.237344 27904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:09.237488 27898 server_base.cc:1034] running on GCE node
W20260806 15:59:09.237375 27905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:09.237978 27898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:09.238663 27898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:09.239861 27898 hybrid_clock.cc:648] HybridClock initialized: now 1786031949239827 us; error 45 us; skew 500 ppm
Aug 06 15:59:09 dist-test-slave-tm1g krb5kdc[27485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031949, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:09.242594 27898 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 15:59:09.243431 27898 webserver.cc:460] Webserver started at http://127.25.31.131:35375/ using document root <none> and password file <none>
I20260806 15:59:09.243992 27898 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:09.244082 27898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:09.244324 27898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:09.246260 27898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "3a20f42ff328424781a4018d5a9064cb"
format_stamp: "Formatted at 2026-08-06 15:59:09 on dist-test-slave-tm1g"
server_key: "fce3cc873ababfff69a7cede53ec0344"
server_key_iv: "d12a89dc693799673ec567570f9fcbfb"
server_key_version: "encryptionkey@0"
I20260806 15:59:09.246883 27898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "3a20f42ff328424781a4018d5a9064cb"
format_stamp: "Formatted at 2026-08-06 15:59:09 on dist-test-slave-tm1g"
server_key: "fce3cc873ababfff69a7cede53ec0344"
server_key_iv: "d12a89dc693799673ec567570f9fcbfb"
server_key_version: "encryptionkey@0"
I20260806 15:59:09.250547 27898 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 15:59:09.253110 27915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:09.254496 27898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 15:59:09.254633 27898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "3a20f42ff328424781a4018d5a9064cb"
format_stamp: "Formatted at 2026-08-06 15:59:09 on dist-test-slave-tm1g"
server_key: "fce3cc873ababfff69a7cede53ec0344"
server_key_iv: "d12a89dc693799673ec567570f9fcbfb"
server_key_version: "encryptionkey@0"
I20260806 15:59:09.254745 27898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:09.284286 27898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:09.287282 27898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:09.287497 27898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:09.288937 27898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:09.289009 27898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:09.289074 27898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:09.289124 27898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 15:59:09 dist-test-slave-tm1g krb5kdc[27485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031949, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:09.299978 27898 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:38759
I20260806 15:59:09.300060 28034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:38759 every 8 connection(s)
I20260806 15:59:09.301026 27898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 15:59:09.301964 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 27898
I20260806 15:59:09.302080 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 15:59:09.302402 25726 external_mini_cluster.cc:1428] Setting key d6c9e6ad109095d5438de4f479c6296e
I20260806 15:59:09.303818 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.289822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:50409 (local address 127.25.31.190:42679)
0806 15:59:09.289982 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.289987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.290892 (+ 905us) server_negotiation.cc:407] Connection header received
0806 15:59:09.291856 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.291859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.291928 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.292023 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:09.293361 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.294018 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.296000 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.296409 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.296537 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.299667 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:09.299672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:09.299677 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:09.299708 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:09.301375 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:09.301981 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:09.301986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:09.301991 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:09.302086 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:09.302532 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:09.302535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:09.302537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:09.302769 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:09.302873 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.303526 (+ 653us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.303645 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 15:59:09.304638 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.290214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42679 (local address 127.25.31.131:50409)
0806 15:59:09.290721 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:09.290770 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:09.291636 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:09.292346 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:09.292357 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:09.292675 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:09.293214 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.293228 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.294176 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.294181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.295821 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.295834 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.296581 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.296586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.296785 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.297973 (+ 1188us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:09.297989 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:09.299498 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:09.301499 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:09.301504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:09.301517 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:09.301863 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:09.302198 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:09.302201 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:09.302203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:09.302403 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:09.302895 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:09.302901 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:09.303234 (+ 333us) client_negotiation.cc:769] Sending connection context
0806 15:59:09.303433 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 15:59:09.303752 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":131,"threads_started":1}
I20260806 15:59:09.305641 28035 heartbeater.cc:347] Connected to a master server at 127.25.31.190:42679
I20260806 15:59:09.305773 28035 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:09.306334 28035 heartbeater.cc:511] Master 127.25.31.190:42679 requested a full tablet report, sending...
I20260806 15:59:09.307360 27533 ts_manager.cc:194] Registered new tserver with Master: 3a20f42ff328424781a4018d5a9064cb (127.25.31.131:38759)
I20260806 15:59:09.308182 27533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:50409
I20260806 15:59:09.314642 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.307925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42679 (local address 127.25.31.131:34341)
0806 15:59:09.308206 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:09.308221 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:09.308312 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:09.308635 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:09.308636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:09.308851 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:09.308964 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.308971 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.309700 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.309703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.310497 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.310506 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.310951 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.310953 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.311020 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.311419 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:09.311429 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:09.311714 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:09.313416 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:09.313418 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:09.313420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:09.313601 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:09.313831 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:09.313833 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:09.313834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:09.313905 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:09.314204 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:09.314205 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:09.314275 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 15:59:09.314410 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 15:59:09.314523 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":81,"threads_started":1}
I20260806 15:59:09.314884 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.307991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:34341 (local address 127.25.31.190:42679)
0806 15:59:09.308104 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.308107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.308285 (+ 178us) server_negotiation.cc:407] Connection header received
0806 15:59:09.308385 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.308387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.308442 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.308523 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:09.309082 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.309570 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.310630 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.310849 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.310940 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.311841 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:09.311844 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:09.311846 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:09.311867 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:09.313296 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:09.313682 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:09.313684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:09.313686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:09.313756 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:09.313977 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:09.313979 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:09.313980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:09.314126 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:09.314200 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.314513 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.314605 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 15:59:09.316229 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 15:59:09 dist-test-slave-tm1g krb5kdc[27485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 15:59:09 dist-test-slave-tm1g krb5kdc[27485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:09.341321 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.333471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42860 (local address 127.25.31.190:42679)
0806 15:59:09.333637 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.333641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.333654 (+ 13us) server_negotiation.cc:407] Connection header received
0806 15:59:09.333715 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.333718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.333766 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.333833 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:09.334554 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.335070 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.335862 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.336068 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.336142 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.338483 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:09.338486 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:09.338488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:09.338510 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:09.339724 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:09.340146 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:09.340149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:09.340151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:09.340239 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:09.340519 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:09.340522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:09.340523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:09.340697 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:09.340809 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.341010 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.341100 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 15:59:09.344231 27533 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:59:09.347273 27533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:09.362021 28055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.355799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:38759 (local address 127.0.0.1:45074)
0806 15:59:09.356570 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:09.356619 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:09.356747 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:09.357352 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:09.357358 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:09.357382 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:09.357537 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.357544 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.358969 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.358972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.360674 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.360681 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.361467 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.361471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.361577 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.361617 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 15:59:09.361740 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 15:59:09.361852 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":200,"thread_start_us":84,"threads_started":1}
I20260806 15:59:09.362218 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.356094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53252 (local address 127.25.31.130:39883)
0806 15:59:09.357086 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.357099 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.357121 (+ 22us) server_negotiation.cc:407] Connection header received
0806 15:59:09.357198 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.357205 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.357391 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.357576 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:09.357827 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.358995 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.360804 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.361640 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.361751 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.361917 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.361968 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.362059 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":877,"thread_start_us":88,"threads_started":1}
I20260806 15:59:09.362465 28058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.356493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:39385 (local address 127.0.0.1:33634)
0806 15:59:09.357235 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:09.357248 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:09.357327 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:09.358346 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:09.358349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:09.358369 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:09.358449 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.358455 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.359769 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.359775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.360673 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.360683 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.362075 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.362080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.362172 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.362193 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 15:59:09.362257 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 15:59:09.362348 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":33,"thread_start_us":78,"threads_started":1}
I20260806 15:59:09.362600 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.356194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45074 (local address 127.25.31.131:38759)
0806 15:59:09.356502 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.356514 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.356891 (+ 377us) server_negotiation.cc:407] Connection header received
0806 15:59:09.356982 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.356991 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.357205 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.357368 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:09.357650 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.358831 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.360808 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.361341 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.361469 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.361635 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.361971 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.362059 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":106,"threads_started":1}
I20260806 15:59:09.362733 28054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.355922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:39883 (local address 127.0.0.1:53252)
0806 15:59:09.356702 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:09.356717 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:09.356798 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:09.357552 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:09.357555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:09.357577 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:09.357694 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.357703 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.359102 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.359105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.360665 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.360676 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.361764 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.361767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.361847 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.361867 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 15:59:09.361927 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 15:59:09.361980 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":107,"thread_start_us":99,"threads_started":1}
I20260806 15:59:09.363644 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.356873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33634 (local address 127.25.31.129:39385)
0806 15:59:09.357923 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.357929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.357951 (+ 22us) server_negotiation.cc:407] Connection header received
0806 15:59:09.358024 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.358028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.358194 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.358359 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:09.358571 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.359639 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.360849 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.361640 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.361751 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.361881 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.363461 (+ 1580us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.363539 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":84,"threads_started":1}
I20260806 15:59:09.365510 27820 tablet_service.cc:1467] Processing CreateTablet for tablet 2109393345ed47e69aa8a77d8a3e045b (DEFAULT_TABLE table=test-table [id=715f889819e1410f9c0e8c5fbff69e36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:09.365509 27967 tablet_service.cc:1467] Processing CreateTablet for tablet 2109393345ed47e69aa8a77d8a3e045b (DEFAULT_TABLE table=test-table [id=715f889819e1410f9c0e8c5fbff69e36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:09.365674 27673 tablet_service.cc:1467] Processing CreateTablet for tablet 2109393345ed47e69aa8a77d8a3e045b (DEFAULT_TABLE table=test-table [id=715f889819e1410f9c0e8c5fbff69e36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:09.366649 27967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2109393345ed47e69aa8a77d8a3e045b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:09.366645 27820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2109393345ed47e69aa8a77d8a3e045b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:09.366699 27673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2109393345ed47e69aa8a77d8a3e045b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:09.373251 28060 tablet_bootstrap.cc:492] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0: Bootstrap starting.
I20260806 15:59:09.374991 28062 tablet_bootstrap.cc:492] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c: Bootstrap starting.
I20260806 15:59:09.375505 28060 tablet_bootstrap.cc:654] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:09.376215 28060 log.cc:824] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:09.376677 28062 tablet_bootstrap.cc:654] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:09.377230 28061 tablet_bootstrap.cc:492] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb: Bootstrap starting.
I20260806 15:59:09.377516 28062 log.cc:824] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:09.379065 28061 tablet_bootstrap.cc:654] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:09.379329 28062 tablet_bootstrap.cc:492] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c: No bootstrap required, opened a new log
I20260806 15:59:09.379522 28062 ts_tablet_manager.cc:1397] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 15:59:09.379822 28061 log.cc:824] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:09.381274 28060 tablet_bootstrap.cc:492] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0: No bootstrap required, opened a new log
I20260806 15:59:09.381479 28060 ts_tablet_manager.cc:1397] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260806 15:59:09.381681 28061 tablet_bootstrap.cc:492] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb: No bootstrap required, opened a new log
I20260806 15:59:09.381861 28061 ts_tablet_manager.cc:1397] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 15:59:09.383188 28062 raft_consensus.cc:348] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } }
I20260806 15:59:09.383452 28062 raft_consensus.cc:374] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:09.383508 28062 raft_consensus.cc:729] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d872c956af834b7c8d8e1237af06f06c, State: Initialized, Role: FOLLOWER
I20260806 15:59:09.383978 28062 consensus_queue.cc:260] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } }
I20260806 15:59:09.384968 28062 ts_tablet_manager.cc:1428] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
I20260806 15:59:09.385185 28060 raft_consensus.cc:348] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } }
I20260806 15:59:09.385340 27889 heartbeater.cc:503] Master 127.25.31.190:42679 was elected leader, sending a full tablet report...
I20260806 15:59:09.385437 28060 raft_consensus.cc:374] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:09.385506 28060 raft_consensus.cc:729] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f76b7d35635404b95797b18318d42c0, State: Initialized, Role: FOLLOWER
I20260806 15:59:09.385900 28060 consensus_queue.cc:260] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } }
I20260806 15:59:09.385784 28061 raft_consensus.cc:348] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } }
I20260806 15:59:09.386037 28061 raft_consensus.cc:374] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:09.386111 28061 raft_consensus.cc:729] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a20f42ff328424781a4018d5a9064cb, State: Initialized, Role: FOLLOWER
I20260806 15:59:09.386579 27741 heartbeater.cc:503] Master 127.25.31.190:42679 was elected leader, sending a full tablet report...
I20260806 15:59:09.386525 28061 consensus_queue.cc:260] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } }
I20260806 15:59:09.386830 28060 ts_tablet_manager.cc:1428] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 15:59:09.387384 28061 ts_tablet_manager.cc:1428] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 15:59:09.387744 28035 heartbeater.cc:503] Master 127.25.31.190:42679 was elected leader, sending a full tablet report...
I20260806 15:59:09.515049 28069 raft_consensus.cc:482] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:09.515247 28069 raft_consensus.cc:504] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } }
I20260806 15:59:09.516467 28069 leader_election.cc:288] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d872c956af834b7c8d8e1237af06f06c (127.25.31.130:39883), 9f76b7d35635404b95797b18318d42c0 (127.25.31.129:39385)
I20260806 15:59:09.520861 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.516695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:39883 (local address 127.25.31.131:51657)
0806 15:59:09.516843 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:09.516859 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:09.516952 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:09.517209 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:09.517211 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:09.517232 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:09.517411 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.517419 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.518544 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.518548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.519511 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.519520 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.520397 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.520401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.520525 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.520551 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 15:59:09.520646 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 15:59:09.520714 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"spinlock_wait_cycles":12032}
I20260806 15:59:09.520910 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.516815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:51657 (local address 127.25.31.130:39883)
0806 15:59:09.516956 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.516961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.516977 (+ 16us) server_negotiation.cc:407] Connection header received
0806 15:59:09.517041 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.517046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.517097 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.517173 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:09.517584 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.518443 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.519641 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.520261 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.520359 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.520400 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.520690 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.520762 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 15:59:09.521458 28070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.516709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:39385 (local address 127.25.31.131:52003)
0806 15:59:09.517187 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:09.517199 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:09.517301 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:09.517582 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:09.517585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:09.517604 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:09.518090 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.518100 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.519094 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.519096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.520035 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.520045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.520973 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.520977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.521100 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.521133 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 15:59:09.521227 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 15:59:09.521294 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":121,"thread_start_us":157,"threads_started":1}
I20260806 15:59:09.521477 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.516871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:52003 (local address 127.25.31.129:39385)
0806 15:59:09.517017 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.517024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.517279 (+ 255us) server_negotiation.cc:407] Connection header received
0806 15:59:09.517391 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.517394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.517447 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.517530 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:09.518223 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.518972 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.520181 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.520826 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.520955 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.521017 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.521252 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.521337 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 15:59:09.521692 27840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2109393345ed47e69aa8a77d8a3e045b" candidate_uuid: "3a20f42ff328424781a4018d5a9064cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d872c956af834b7c8d8e1237af06f06c" is_pre_election: true
I20260806 15:59:09.522038 27840 raft_consensus.cc:2454] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a20f42ff328424781a4018d5a9064cb in term 0.
I20260806 15:59:09.522166 27693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2109393345ed47e69aa8a77d8a3e045b" candidate_uuid: "3a20f42ff328424781a4018d5a9064cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f76b7d35635404b95797b18318d42c0" is_pre_election: true
I20260806 15:59:09.522493 27693 raft_consensus.cc:2454] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a20f42ff328424781a4018d5a9064cb in term 0.
I20260806 15:59:09.522555 27917 leader_election.cc:302] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a20f42ff328424781a4018d5a9064cb, d872c956af834b7c8d8e1237af06f06c; no voters:
I20260806 15:59:09.522856 28069 raft_consensus.cc:2781] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:09.522960 28069 raft_consensus.cc:482] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:09.523015 28069 raft_consensus.cc:3037] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:09.524098 28069 raft_consensus.cc:504] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } }
I20260806 15:59:09.524427 28069 leader_election.cc:288] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [CANDIDATE]: Term 1 election: Requested vote from peers d872c956af834b7c8d8e1237af06f06c (127.25.31.130:39883), 9f76b7d35635404b95797b18318d42c0 (127.25.31.129:39385)
I20260806 15:59:09.524729 27840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2109393345ed47e69aa8a77d8a3e045b" candidate_uuid: "3a20f42ff328424781a4018d5a9064cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d872c956af834b7c8d8e1237af06f06c"
I20260806 15:59:09.524820 27693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2109393345ed47e69aa8a77d8a3e045b" candidate_uuid: "3a20f42ff328424781a4018d5a9064cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f76b7d35635404b95797b18318d42c0"
I20260806 15:59:09.524873 27840 raft_consensus.cc:3037] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:09.524971 27693 raft_consensus.cc:3037] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:09.525969 27840 raft_consensus.cc:2454] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a20f42ff328424781a4018d5a9064cb in term 1.
I20260806 15:59:09.526048 27693 raft_consensus.cc:2454] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a20f42ff328424781a4018d5a9064cb in term 1.
I20260806 15:59:09.526280 27917 leader_election.cc:302] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a20f42ff328424781a4018d5a9064cb, d872c956af834b7c8d8e1237af06f06c; no voters:
I20260806 15:59:09.526465 28069 raft_consensus.cc:2781] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:09.526688 28069 raft_consensus.cc:686] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [term 1 LEADER]: Becoming Leader. State: Replica: 3a20f42ff328424781a4018d5a9064cb, State: Running, Role: LEADER
I20260806 15:59:09.527005 28069 consensus_queue.cc:237] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } }
I20260806 15:59:09.530428 27530 catalog_manager.cc:5495] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a20f42ff328424781a4018d5a9064cb (127.25.31.131). New cstate: current_term: 1 leader_uuid: "3a20f42ff328424781a4018d5a9064cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } health_report { overall_health: UNKNOWN } } }
W20260806 15:59:09.552175 28036 tablet.cc:2369] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 15:59:09 dist-test-slave-tm1g krb5kdc[27485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031948, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:09.567967 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.559843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42679 (local address 127.0.0.1:42876)
0806 15:59:09.560162 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:09.560175 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:09.560319 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:09.560577 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:09.560580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:09.560969 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:09.561057 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.561065 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.561829 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.561835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.562414 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.562422 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.562862 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.562865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.562950 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.563487 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:09.563499 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:09.564862 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:09.566613 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:09.566622 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:09.566627 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:09.566854 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:09.567096 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:09.567099 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:09.567100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:09.567182 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:09.567517 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:09.567522 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:09.567623 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 15:59:09.567719 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 15:59:09.567841 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":111,"threads_started":1}
W20260806 15:59:09.568048 27890 tablet.cc:2369] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:59:09.568116 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.559930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42876 (local address 127.25.31.190:42679)
0806 15:59:09.560088 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.560093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.560251 (+ 158us) server_negotiation.cc:407] Connection header received
0806 15:59:09.560417 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.560422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.560470 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.560554 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:09.561182 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.561714 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.562535 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.562748 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.562863 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.565007 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:09.565010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:09.565012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:09.565033 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:09.566518 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:09.566938 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:09.566941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:09.566942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:09.567011 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:09.567261 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:09.567263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:09.567264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:09.567420 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:09.567493 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.567772 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.567942 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":52}
I20260806 15:59:09.571933 27530 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:42876:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 15:59:09.572671 27530 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:09.577584 27820 tablet_service.cc:1467] Processing CreateTablet for tablet 55b2793737a74ed68d8e1e6100252dec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f309d739d5c94e76b2b7a43b7d567875]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:59:09.577579 27673 tablet_service.cc:1467] Processing CreateTablet for tablet 55b2793737a74ed68d8e1e6100252dec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f309d739d5c94e76b2b7a43b7d567875]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:59:09.577894 27967 tablet_service.cc:1467] Processing CreateTablet for tablet 55b2793737a74ed68d8e1e6100252dec (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f309d739d5c94e76b2b7a43b7d567875]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:59:09.578286 27673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55b2793737a74ed68d8e1e6100252dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:09.578259 27820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55b2793737a74ed68d8e1e6100252dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:09.578474 27967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55b2793737a74ed68d8e1e6100252dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:09.580746 28062 tablet_bootstrap.cc:492] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c: Bootstrap starting.
I20260806 15:59:09.581197 28060 tablet_bootstrap.cc:492] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0: Bootstrap starting.
I20260806 15:59:09.581826 28062 tablet_bootstrap.cc:654] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:09.582129 28060 tablet_bootstrap.cc:654] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:09.582794 28061 tablet_bootstrap.cc:492] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb: Bootstrap starting.
I20260806 15:59:09.583298 28062 tablet_bootstrap.cc:492] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c: No bootstrap required, opened a new log
I20260806 15:59:09.583380 28062 ts_tablet_manager.cc:1397] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 15:59:09.583696 28061 tablet_bootstrap.cc:654] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:09.583801 28062 raft_consensus.cc:348] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:09.583920 28062 raft_consensus.cc:374] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:09.583945 28062 raft_consensus.cc:729] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d872c956af834b7c8d8e1237af06f06c, State: Initialized, Role: FOLLOWER
I20260806 15:59:09.584074 28062 consensus_queue.cc:260] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:09.584396 28062 ts_tablet_manager.cc:1428] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 15:59:09.584404 28066 tablet_replica.cc:439] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } }
I20260806 15:59:09.584594 28066 tablet_replica.cc:442] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:09.584892 28060 tablet_bootstrap.cc:492] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0: No bootstrap required, opened a new log
I20260806 15:59:09.584985 28060 ts_tablet_manager.cc:1397] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 15:59:09.585438 28061 tablet_bootstrap.cc:492] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb: No bootstrap required, opened a new log
I20260806 15:59:09.585536 28061 ts_tablet_manager.cc:1397] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 15:59:09.585484 28060 raft_consensus.cc:348] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:09.585587 28060 raft_consensus.cc:374] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:09.585636 28060 raft_consensus.cc:729] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f76b7d35635404b95797b18318d42c0, State: Initialized, Role: FOLLOWER
I20260806 15:59:09.585769 28060 consensus_queue.cc:260] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:09.586046 28060 ts_tablet_manager.cc:1428] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:59:09.585994 28061 raft_consensus.cc:348] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:09.586097 28061 raft_consensus.cc:374] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:09.586134 28061 raft_consensus.cc:729] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a20f42ff328424781a4018d5a9064cb, State: Initialized, Role: FOLLOWER
I20260806 15:59:09.586289 28061 consensus_queue.cc:260] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:09.586339 28068 tablet_replica.cc:439] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } }
I20260806 15:59:09.586520 28068 tablet_replica.cc:442] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:09.586581 28069 tablet_replica.cc:439] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } }
I20260806 15:59:09.586709 28069 tablet_replica.cc:442] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:09.586588 28061 ts_tablet_manager.cc:1428] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20260806 15:59:09.600432 27742 tablet.cc:2369] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:59:09.768747 28068 raft_consensus.cc:482] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:09.768887 28068 raft_consensus.cc:504] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:09.770079 28068 leader_election.cc:288] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d872c956af834b7c8d8e1237af06f06c (127.25.31.130:39883), 3a20f42ff328424781a4018d5a9064cb (127.25.31.131:38759)
I20260806 15:59:09.774657 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.770705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:32881 (local address 127.25.31.131:38759)
0806 15:59:09.770891 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.770898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.770924 (+ 26us) server_negotiation.cc:407] Connection header received
0806 15:59:09.771068 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.771070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.771120 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.771205 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:09.771747 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.772296 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.773459 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.773976 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.774087 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.774145 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.774434 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.774522 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 15:59:09.774708 28083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.770375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:38759 (local address 127.25.31.129:32881)
0806 15:59:09.770766 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:09.770827 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:09.770937 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:09.771231 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:09.771234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:09.771273 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:09.771609 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.771618 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.772421 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.772426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.773304 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.773316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.774145 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.774149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.774256 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.774290 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 15:59:09.774413 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 15:59:09.774508 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":58,"spinlock_wait_cycles":29184,"thread_start_us":129,"threads_started":1}
I20260806 15:59:09.774904 28082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.770399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:39883 (local address 127.25.31.129:58217)
0806 15:59:09.770895 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:09.770907 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:09.770994 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:09.771231 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:09.771234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:09.771275 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:09.771972 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.771980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.772750 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.772756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.773588 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.773599 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.774418 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.774421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.774555 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.774578 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 15:59:09.774655 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 15:59:09.774729 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":163,"spinlock_wait_cycles":7808,"thread_start_us":89,"threads_started":1}
I20260806 15:59:09.774935 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.770596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:58217 (local address 127.25.31.130:39883)
0806 15:59:09.770729 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.770734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.771005 (+ 271us) server_negotiation.cc:407] Connection header received
0806 15:59:09.771067 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.771070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.771120 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.771205 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:09.772128 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.772638 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.773716 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.774256 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.774434 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.774482 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.774690 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.774785 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 15:59:09.775338 27840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b2793737a74ed68d8e1e6100252dec" candidate_uuid: "9f76b7d35635404b95797b18318d42c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d872c956af834b7c8d8e1237af06f06c" is_pre_election: true
I20260806 15:59:09.775380 27988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b2793737a74ed68d8e1e6100252dec" candidate_uuid: "9f76b7d35635404b95797b18318d42c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a20f42ff328424781a4018d5a9064cb" is_pre_election: true
I20260806 15:59:09.775472 27840 raft_consensus.cc:2454] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f76b7d35635404b95797b18318d42c0 in term 0.
I20260806 15:59:09.775641 27988 raft_consensus.cc:2454] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f76b7d35635404b95797b18318d42c0 in term 0.
I20260806 15:59:09.775854 27624 leader_election.cc:302] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f76b7d35635404b95797b18318d42c0, d872c956af834b7c8d8e1237af06f06c; no voters:
I20260806 15:59:09.776165 28068 raft_consensus.cc:2781] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:09.776232 28068 raft_consensus.cc:482] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:09.776258 28068 raft_consensus.cc:3037] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:09.777272 28068 raft_consensus.cc:504] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:09.777631 28068 leader_election.cc:288] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [CANDIDATE]: Term 1 election: Requested vote from peers d872c956af834b7c8d8e1237af06f06c (127.25.31.130:39883), 3a20f42ff328424781a4018d5a9064cb (127.25.31.131:38759)
I20260806 15:59:09.777951 27988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b2793737a74ed68d8e1e6100252dec" candidate_uuid: "9f76b7d35635404b95797b18318d42c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a20f42ff328424781a4018d5a9064cb"
I20260806 15:59:09.777952 27840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55b2793737a74ed68d8e1e6100252dec" candidate_uuid: "9f76b7d35635404b95797b18318d42c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d872c956af834b7c8d8e1237af06f06c"
I20260806 15:59:09.778077 27988 raft_consensus.cc:3037] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:09.778077 27840 raft_consensus.cc:3037] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:09.779250 27840 raft_consensus.cc:2454] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f76b7d35635404b95797b18318d42c0 in term 1.
I20260806 15:59:09.779250 27988 raft_consensus.cc:2454] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f76b7d35635404b95797b18318d42c0 in term 1.
I20260806 15:59:09.779549 27624 leader_election.cc:302] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f76b7d35635404b95797b18318d42c0, d872c956af834b7c8d8e1237af06f06c; no voters:
I20260806 15:59:09.779726 28068 raft_consensus.cc:2781] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:09.779958 28068 raft_consensus.cc:686] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [term 1 LEADER]: Becoming Leader. State: Replica: 9f76b7d35635404b95797b18318d42c0, State: Running, Role: LEADER
I20260806 15:59:09.780295 28068 consensus_queue.cc:237] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:09.782459 28084 tablet_replica.cc:439] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0: TxnStatusTablet state changed. Reason: New leader 9f76b7d35635404b95797b18318d42c0. Latest consensus state: current_term: 1 leader_uuid: "9f76b7d35635404b95797b18318d42c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } }
I20260806 15:59:09.782567 28084 tablet_replica.cc:442] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:59:09.782841 28086 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 15:59:09.783059 28086 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 15:59:09.783425 27530 catalog_manager.cc:5495] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f76b7d35635404b95797b18318d42c0 (127.25.31.129). New cstate: current_term: 1 leader_uuid: "9f76b7d35635404b95797b18318d42c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } health_report { overall_health: UNKNOWN } } }
I20260806 15:59:09.824638 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.820485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:39385 (local address 127.0.0.1:33640)
0806 15:59:09.820645 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:09.820663 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:09.820771 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:09.821129 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:09.821132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:09.821138 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:59:09.821238 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.821242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.822253 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.822256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.822988 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.822998 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.823554 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.823557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.823623 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.823673 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:59:09.824399 (+ 726us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:59:09.824410 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 15:59:09.824457 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 15:59:09.824525 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 15:59:09.824658 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.820621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33640 (local address 127.25.31.129:39385)
0806 15:59:09.820740 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.820746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.820761 (+ 15us) server_negotiation.cc:407] Connection header received
0806 15:59:09.820932 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.820935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.821000 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.821112 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:09.821345 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.822129 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.823105 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.823451 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.823554 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.823809 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:09.824304 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:09.824409 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.824472 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.824524 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 15:59:09.829739 27840 raft_consensus.cc:1261] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Refusing update from remote peer 9f76b7d35635404b95797b18318d42c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:09.829823 27988 raft_consensus.cc:1261] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Refusing update from remote peer 9f76b7d35635404b95797b18318d42c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:09.830461 28084 consensus_queue.cc:1035] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:09.830719 28068 consensus_queue.cc:1035] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:09.833698 28069 tablet_replica.cc:439] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb: TxnStatusTablet state changed. Reason: New leader 9f76b7d35635404b95797b18318d42c0. Latest consensus state: current_term: 1 leader_uuid: "9f76b7d35635404b95797b18318d42c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } }
I20260806 15:59:09.833807 28069 tablet_replica.cc:442] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:09.834990 28066 tablet_replica.cc:439] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c: TxnStatusTablet state changed. Reason: New leader 9f76b7d35635404b95797b18318d42c0. Latest consensus state: current_term: 1 leader_uuid: "9f76b7d35635404b95797b18318d42c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } }
I20260806 15:59:09.835114 28066 tablet_replica.cc:442] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:09.838441 28068 tablet_replica.cc:439] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f76b7d35635404b95797b18318d42c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } }
I20260806 15:59:09.838567 28068 tablet_replica.cc:442] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:59:09.838694 28068 tablet_replica.cc:439] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9f76b7d35635404b95797b18318d42c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } }
I20260806 15:59:09.838783 28068 tablet_replica.cc:442] T 55b2793737a74ed68d8e1e6100252dec P 9f76b7d35635404b95797b18318d42c0: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:59:09.839092 28091 mvcc.cc:205] Tried to move back new op lower bound from 7315586866496040960 to 7315586866302382080. Current Snapshot: MvccSnapshot[applied={T|T < 7315586866496040960}]
I20260806 15:59:09.839082 28066 tablet_replica.cc:439] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f76b7d35635404b95797b18318d42c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } }
I20260806 15:59:09.839177 28066 tablet_replica.cc:442] T 55b2793737a74ed68d8e1e6100252dec P d872c956af834b7c8d8e1237af06f06c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:09.839236 28069 tablet_replica.cc:439] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9f76b7d35635404b95797b18318d42c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } } }
I20260806 15:59:09.839319 28069 tablet_replica.cc:442] T 55b2793737a74ed68d8e1e6100252dec P 3a20f42ff328424781a4018d5a9064cb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:09.839928 28090 mvcc.cc:205] Tried to move back new op lower bound from 7315586866496040960 to 7315586866302382080. Current Snapshot: MvccSnapshot[applied={T|T < 7315586866496040960}]
I20260806 15:59:09.840015 28089 mvcc.cc:205] Tried to move back new op lower bound from 7315586866496040960 to 7315586866302382080. Current Snapshot: MvccSnapshot[applied={T|T < 7315586866496040960}]
I20260806 15:59:09.850791 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.846111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45078 (local address 127.25.31.131:38759)
0806 15:59:09.846243 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.846248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.846502 (+ 254us) server_negotiation.cc:407] Connection header received
0806 15:59:09.846619 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.846622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.846680 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.846786 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:09.847070 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.847862 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.849099 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.849479 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.849612 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.849926 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:09.850423 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:09.850526 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.850613 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.850668 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 15:59:09.866747 28102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.862812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:39385 (local address 127.25.31.131:32961)
0806 15:59:09.863305 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:09.863320 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:09.863433 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:09.863680 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:09.863687 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:09.863700 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:59:09.863828 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.863834 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.864820 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.864822 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.865468 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.865476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.866044 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.866047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.866128 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.866170 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:59:09.866507 (+ 337us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:59:09.866513 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 15:59:09.866569 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 15:59:09.866631 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":92,"threads_started":1}
I20260806 15:59:09.866755 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.863111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:32961 (local address 127.25.31.129:39385)
0806 15:59:09.863239 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.863244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.863405 (+ 161us) server_negotiation.cc:407] Connection header received
0806 15:59:09.863526 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.863530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.863576 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.863649 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:09.863940 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.864708 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.865578 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.865923 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.866026 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.866335 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:09.866421 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:09.866494 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.866580 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.866632 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 15:59:09.873322 27840 raft_consensus.cc:1261] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Refusing update from remote peer 3a20f42ff328424781a4018d5a9064cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:09.873396 27693 raft_consensus.cc:1261] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Refusing update from remote peer 3a20f42ff328424781a4018d5a9064cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:09.873806 28071 consensus_queue.cc:1035] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:09.874136 28071 consensus_queue.cc:1035] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:09.897262 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.892824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:38755 (local address 127.25.31.131:38759)
0806 15:59:09.892931 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:09.892935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:09.893234 (+ 299us) server_negotiation.cc:407] Connection header received
0806 15:59:09.893461 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:09.893464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:09.893532 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:09.893605 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:09.893990 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.894674 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.896094 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.896378 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.896496 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.896819 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:09.896906 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:09.896976 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:09.897066 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 15:59:09.897131 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 15:59:09.897290 28106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:09.892758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:38759 (local address 127.25.31.129:38755)
0806 15:59:09.893127 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:09.893143 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:09.893253 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:09.893627 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:09.893629 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:09.893637 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:59:09.893774 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.893780 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.894845 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.894849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.895966 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:09.895977 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:09.896497 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:09.896501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:09.896613 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:09.896672 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:59:09.896979 (+ 307us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:59:09.896985 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 15:59:09.897048 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 15:59:09.897134 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":125,"threads_started":1}
I20260806 15:59:09.920667 27530 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:09.920866 27530 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:09.923583 27530 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e: Sending DeleteTablet for 3 replicas of tablet 2109393345ed47e69aa8a77d8a3e045b
I20260806 15:59:09.924358 27967 tablet_service.cc:1514] Processing DeleteTablet for tablet 2109393345ed47e69aa8a77d8a3e045b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:09 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:45074
I20260806 15:59:09.924522 27820 tablet_service.cc:1514] Processing DeleteTablet for tablet 2109393345ed47e69aa8a77d8a3e045b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:09 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:53252
I20260806 15:59:09.924983 28110 tablet_replica.cc:330] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb: stopping tablet replica
I20260806 15:59:09.925348 28110 raft_consensus.cc:2229] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:59:09.925757 28110 raft_consensus.cc:2258] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:09.925861 27673 tablet_service.cc:1514] Processing DeleteTablet for tablet 2109393345ed47e69aa8a77d8a3e045b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:09 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:33634
I20260806 15:59:09.926386 27530 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:59:09.926801 27530 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:09.927481 28110 ts_tablet_manager.cc:1905] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:09.928342 28111 tablet_replica.cc:330] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c: stopping tablet replica
I20260806 15:59:09.928643 28111 raft_consensus.cc:2229] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:09.928833 28111 raft_consensus.cc:2258] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:09.930053 28111 ts_tablet_manager.cc:1905] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:09.932843 27673 tablet_service.cc:1467] Processing CreateTablet for tablet ae251ba45c1a4642820db5f455ce6c35 (DEFAULT_TABLE table=test-table [id=154b3709af6149c1b788b308ea6946e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:09.933043 27820 tablet_service.cc:1467] Processing CreateTablet for tablet ae251ba45c1a4642820db5f455ce6c35 (DEFAULT_TABLE table=test-table [id=154b3709af6149c1b788b308ea6946e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:09.933149 27673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae251ba45c1a4642820db5f455ce6c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:09.933276 27820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae251ba45c1a4642820db5f455ce6c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:09.934860 27967 tablet_service.cc:1467] Processing CreateTablet for tablet ae251ba45c1a4642820db5f455ce6c35 (DEFAULT_TABLE table=test-table [id=154b3709af6149c1b788b308ea6946e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:09.935400 28062 tablet_bootstrap.cc:492] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c: Bootstrap starting.
I20260806 15:59:09.936362 28062 tablet_bootstrap.cc:654] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:09.936643 27967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae251ba45c1a4642820db5f455ce6c35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:09.936972 28060 tablet_bootstrap.cc:492] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0: Bootstrap starting.
I20260806 15:59:09.937968 28060 tablet_bootstrap.cc:654] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:09.938910 28061 tablet_bootstrap.cc:492] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb: Bootstrap starting.
I20260806 15:59:09.939548 28110 ts_tablet_manager.cc:1918] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:59:09.939630 28110 log.cc:1196] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/2109393345ed47e69aa8a77d8a3e045b
I20260806 15:59:09.939815 28061 tablet_bootstrap.cc:654] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:09.939917 28110 ts_tablet_manager.cc:1939] T 2109393345ed47e69aa8a77d8a3e045b P 3a20f42ff328424781a4018d5a9064cb: Deleting consensus metadata
I20260806 15:59:09.940201 28112 tablet_replica.cc:330] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0: stopping tablet replica
I20260806 15:59:09.940483 28112 raft_consensus.cc:2229] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:09.940629 28112 raft_consensus.cc:2258] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:09.942366 27515 catalog_manager.cc:4841] TS 3a20f42ff328424781a4018d5a9064cb (127.25.31.131:38759): tablet 2109393345ed47e69aa8a77d8a3e045b (table test-table [id=715f889819e1410f9c0e8c5fbff69e36]) successfully deleted
I20260806 15:59:09.942478 28112 ts_tablet_manager.cc:1905] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:09.942937 28111 ts_tablet_manager.cc:1918] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:59:09.943199 28111 log.cc:1196] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/2109393345ed47e69aa8a77d8a3e045b
I20260806 15:59:09.943586 28111 ts_tablet_manager.cc:1939] T 2109393345ed47e69aa8a77d8a3e045b P d872c956af834b7c8d8e1237af06f06c: Deleting consensus metadata
I20260806 15:59:09.944788 27517 catalog_manager.cc:4841] TS d872c956af834b7c8d8e1237af06f06c (127.25.31.130:39883): tablet 2109393345ed47e69aa8a77d8a3e045b (table test-table [id=715f889819e1410f9c0e8c5fbff69e36]) successfully deleted
I20260806 15:59:09.946135 28112 ts_tablet_manager.cc:1918] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:59:09.946229 28112 log.cc:1196] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/wal/wals/2109393345ed47e69aa8a77d8a3e045b
I20260806 15:59:09.946584 28112 ts_tablet_manager.cc:1939] T 2109393345ed47e69aa8a77d8a3e045b P 9f76b7d35635404b95797b18318d42c0: Deleting consensus metadata
I20260806 15:59:09.946996 28062 tablet_bootstrap.cc:492] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c: No bootstrap required, opened a new log
I20260806 15:59:09.947037 28061 tablet_bootstrap.cc:492] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb: No bootstrap required, opened a new log
I20260806 15:59:09.947036 28060 tablet_bootstrap.cc:492] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0: No bootstrap required, opened a new log
I20260806 15:59:09.947081 28062 ts_tablet_manager.cc:1397] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260806 15:59:09.947115 28061 ts_tablet_manager.cc:1397] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260806 15:59:09.947115 28060 ts_tablet_manager.cc:1397] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20260806 15:59:09.947503 28062 raft_consensus.cc:348] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:09.947553 28061 raft_consensus.cc:348] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:09.947552 28060 raft_consensus.cc:348] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:09.947629 28062 raft_consensus.cc:374] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:09.947659 28061 raft_consensus.cc:374] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:09.947659 28060 raft_consensus.cc:374] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:09.947669 28062 raft_consensus.cc:729] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d872c956af834b7c8d8e1237af06f06c, State: Initialized, Role: FOLLOWER
I20260806 15:59:09.947651 27515 catalog_manager.cc:4841] TS 9f76b7d35635404b95797b18318d42c0 (127.25.31.129:39385): tablet 2109393345ed47e69aa8a77d8a3e045b (table test-table [id=715f889819e1410f9c0e8c5fbff69e36]) successfully deleted
I20260806 15:59:09.947695 28060 raft_consensus.cc:729] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f76b7d35635404b95797b18318d42c0, State: Initialized, Role: FOLLOWER
I20260806 15:59:09.947695 28061 raft_consensus.cc:729] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a20f42ff328424781a4018d5a9064cb, State: Initialized, Role: FOLLOWER
I20260806 15:59:09.947813 28061 consensus_queue.cc:260] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:09.947813 28060 consensus_queue.cc:260] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:09.947798 28062 consensus_queue.cc:260] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:09.948066 28061 ts_tablet_manager.cc:1428] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 15:59:09.948066 28060 ts_tablet_manager.cc:1428] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 15:59:09.948107 28062 ts_tablet_manager.cc:1428] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:59:10.030689 28066 raft_consensus.cc:482] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:10.030900 28066 raft_consensus.cc:504] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:10.032193 28066 leader_election.cc:288] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f76b7d35635404b95797b18318d42c0 (127.25.31.129:39385), 3a20f42ff328424781a4018d5a9064cb (127.25.31.131:38759)
I20260806 15:59:10.037076 28117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:10.032827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:38759 (local address 127.25.31.130:41495)
0806 15:59:10.033093 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:10.033109 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:10.033203 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:10.033568 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:10.033571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:10.033594 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:10.033740 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:10.033750 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.034739 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.034744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:10.035724 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:10.035734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.036611 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.036615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:10.036722 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:10.036749 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 15:59:10.036834 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 15:59:10.036903 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":39,"thread_start_us":78,"threads_started":1}
I20260806 15:59:10.037102 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:10.032947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:41495 (local address 127.25.31.131:38759)
0806 15:59:10.033074 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:10.033079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:10.033199 (+ 120us) server_negotiation.cc:407] Connection header received
0806 15:59:10.033296 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:10.033301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:10.033358 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:10.033441 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:10.033878 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.034629 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.035845 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.036464 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.036586 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:10.036644 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:10.036864 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 15:59:10.036961 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 15:59:10.037185 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:10.032533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:55615 (local address 127.25.31.129:39385)
0806 15:59:10.032676 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:10.032680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:10.033112 (+ 432us) server_negotiation.cc:407] Connection header received
0806 15:59:10.033183 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:10.033187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:10.033249 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:10.033332 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:10.033666 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.034472 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.036020 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.036617 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.036723 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:10.036771 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:10.036991 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 15:59:10.037064 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 15:59:10.037559 28116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:10.032404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:39385 (local address 127.25.31.130:55615)
0806 15:59:10.032700 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:10.032714 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:10.032834 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:10.033346 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:10.033349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:10.033362 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:10.033490 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:10.033498 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.034591 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.034595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:10.035889 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:10.035901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.036722 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.036726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:10.036851 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:10.036874 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 15:59:10.037315 (+ 441us) client_negotiation.cc:240] Negotiation successful
0806 15:59:10.037421 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":32,"thread_start_us":124,"threads_started":1}
I20260806 15:59:10.037699 27988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae251ba45c1a4642820db5f455ce6c35" candidate_uuid: "d872c956af834b7c8d8e1237af06f06c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a20f42ff328424781a4018d5a9064cb" is_pre_election: true
I20260806 15:59:10.037860 27988 raft_consensus.cc:2454] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d872c956af834b7c8d8e1237af06f06c in term 0.
I20260806 15:59:10.038231 27767 leader_election.cc:302] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a20f42ff328424781a4018d5a9064cb, d872c956af834b7c8d8e1237af06f06c; no voters:
I20260806 15:59:10.038384 27693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae251ba45c1a4642820db5f455ce6c35" candidate_uuid: "d872c956af834b7c8d8e1237af06f06c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f76b7d35635404b95797b18318d42c0" is_pre_election: true
I20260806 15:59:10.038513 28066 raft_consensus.cc:2781] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:10.038529 27693 raft_consensus.cc:2454] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d872c956af834b7c8d8e1237af06f06c in term 0.
I20260806 15:59:10.038573 28066 raft_consensus.cc:482] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:10.038609 28066 raft_consensus.cc:3037] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:10.039664 28066 raft_consensus.cc:504] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:10.039968 28066 leader_election.cc:288] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [CANDIDATE]: Term 1 election: Requested vote from peers 9f76b7d35635404b95797b18318d42c0 (127.25.31.129:39385), 3a20f42ff328424781a4018d5a9064cb (127.25.31.131:38759)
I20260806 15:59:10.040385 27693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae251ba45c1a4642820db5f455ce6c35" candidate_uuid: "d872c956af834b7c8d8e1237af06f06c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f76b7d35635404b95797b18318d42c0"
I20260806 15:59:10.040417 27988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae251ba45c1a4642820db5f455ce6c35" candidate_uuid: "d872c956af834b7c8d8e1237af06f06c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a20f42ff328424781a4018d5a9064cb"
I20260806 15:59:10.040514 27693 raft_consensus.cc:3037] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:10.040536 27988 raft_consensus.cc:3037] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:10.041417 27693 raft_consensus.cc:2454] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d872c956af834b7c8d8e1237af06f06c in term 1.
I20260806 15:59:10.041699 27767 leader_election.cc:302] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f76b7d35635404b95797b18318d42c0, d872c956af834b7c8d8e1237af06f06c; no voters:
I20260806 15:59:10.041750 27988 raft_consensus.cc:2454] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d872c956af834b7c8d8e1237af06f06c in term 1.
I20260806 15:59:10.041915 28066 raft_consensus.cc:2781] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:10.042143 28066 raft_consensus.cc:686] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [term 1 LEADER]: Becoming Leader. State: Replica: d872c956af834b7c8d8e1237af06f06c, State: Running, Role: LEADER
I20260806 15:59:10.042440 28066 consensus_queue.cc:237] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:10.044884 27532 catalog_manager.cc:5495] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c reported cstate change: term changed from 0 to 1, leader changed from <none> to d872c956af834b7c8d8e1237af06f06c (127.25.31.130). New cstate: current_term: 1 leader_uuid: "d872c956af834b7c8d8e1237af06f06c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } health_report { overall_health: UNKNOWN } } }
I20260806 15:59:10.074542 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:10.070057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53258 (local address 127.25.31.130:39883)
0806 15:59:10.070162 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:10.070166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:10.070235 (+ 69us) server_negotiation.cc:407] Connection header received
0806 15:59:10.070438 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:10.070441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:10.070499 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:10.070590 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:10.070877 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.071666 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.072852 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.073210 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.073296 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:10.073649 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:10.074120 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:10.074230 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:10.074361 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 15:59:10.074423 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 15:59:10.086429 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:10.082195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:39385 (local address 127.25.31.130:35325)
0806 15:59:10.082505 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:10.082520 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:10.082619 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:10.082862 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:10.082864 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:10.082871 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:59:10.082962 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:10.082967 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.083992 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.083995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:10.084953 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:10.084961 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.085627 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.085630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:10.085730 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:10.085783 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:59:10.086121 (+ 338us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:59:10.086126 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:59:10.086176 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 15:59:10.086235 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":125,"threads_started":1}
I20260806 15:59:10.086437 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:10.082300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:35325 (local address 127.25.31.129:39385)
0806 15:59:10.082492 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:10.082497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:10.082606 (+ 109us) server_negotiation.cc:407] Connection header received
0806 15:59:10.082706 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:10.082709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:10.082762 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:10.082839 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:10.083080 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.083862 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.085096 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.085503 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.085615 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:10.085942 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:10.086032 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:10.086112 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:10.086212 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 15:59:10.086269 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 15:59:10.093115 27988 raft_consensus.cc:1261] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Refusing update from remote peer d872c956af834b7c8d8e1237af06f06c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:10.093230 27693 raft_consensus.cc:1261] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Refusing update from remote peer d872c956af834b7c8d8e1237af06f06c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:10.093644 28066 consensus_queue.cc:1035] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:10.093813 28118 consensus_queue.cc:1035] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:10.112017 28106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:10.108318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:39883 (local address 127.25.31.129:56223)
0806 15:59:10.108519 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:10.108535 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:10.108624 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:10.108875 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:10.108878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:10.108886 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:59:10.108990 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:10.108995 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.109898 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.109902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:10.110583 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:10.110592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.111160 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.111163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:10.111375 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:10.111410 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:59:10.111690 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:59:10.111699 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 15:59:10.111796 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 15:59:10.111891 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 15:59:10.112017 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:10.108394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:56223 (local address 127.25.31.130:39883)
0806 15:59:10.108516 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:10.108520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:10.108579 (+ 59us) server_negotiation.cc:407] Connection header received
0806 15:59:10.108731 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:10.108734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:10.108777 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:10.108864 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:10.109100 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.109788 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.110701 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:10.111015 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:10.111142 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:10.111530 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:10.111596 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:10.111682 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:10.111831 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 15:59:10.111882 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 15:59:10.112727 28090 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 15:59:11.667726 27532 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:11.667879 27532 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:11.670205 27532 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e: Sending DeleteTablet for 3 replicas of tablet ae251ba45c1a4642820db5f455ce6c35
I20260806 15:59:11.670749 27673 tablet_service.cc:1514] Processing DeleteTablet for tablet ae251ba45c1a4642820db5f455ce6c35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:11 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:33634
I20260806 15:59:11.670871 27967 tablet_service.cc:1514] Processing DeleteTablet for tablet ae251ba45c1a4642820db5f455ce6c35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:11 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:45074
I20260806 15:59:11.670871 27820 tablet_service.cc:1514] Processing DeleteTablet for tablet ae251ba45c1a4642820db5f455ce6c35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:11 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:53252
I20260806 15:59:11.671201 28151 tablet_replica.cc:330] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0: stopping tablet replica
I20260806 15:59:11.671844 28152 tablet_replica.cc:330] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c: stopping tablet replica
I20260806 15:59:11.671855 28151 raft_consensus.cc:2229] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:11.671980 28151 raft_consensus.cc:2258] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:11.672008 28152 raft_consensus.cc:2229] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:59:11.672317 28152 raft_consensus.cc:2258] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:11.671182 28153 tablet_replica.cc:330] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb: stopping tablet replica
I20260806 15:59:11.672597 28153 raft_consensus.cc:2229] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:11.672644 28151 ts_tablet_manager.cc:1905] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:11.672727 28153 raft_consensus.cc:2258] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:11.673441 28153 ts_tablet_manager.cc:1905] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:11.674485 28152 ts_tablet_manager.cc:1905] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:11.675396 28151 ts_tablet_manager.cc:1918] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 15:59:11.675489 28151 log.cc:1196] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/wal/wals/ae251ba45c1a4642820db5f455ce6c35
I20260806 15:59:11.675731 28151 ts_tablet_manager.cc:1939] T ae251ba45c1a4642820db5f455ce6c35 P 9f76b7d35635404b95797b18318d42c0: Deleting consensus metadata
I20260806 15:59:11.676198 28153 ts_tablet_manager.cc:1918] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 15:59:11.676293 28153 log.cc:1196] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/ae251ba45c1a4642820db5f455ce6c35
I20260806 15:59:11.676371 27515 catalog_manager.cc:4841] TS 9f76b7d35635404b95797b18318d42c0 (127.25.31.129:39385): tablet ae251ba45c1a4642820db5f455ce6c35 (table test-table [id=154b3709af6149c1b788b308ea6946e0]) successfully deleted
I20260806 15:59:11.676568 28153 ts_tablet_manager.cc:1939] T ae251ba45c1a4642820db5f455ce6c35 P 3a20f42ff328424781a4018d5a9064cb: Deleting consensus metadata
I20260806 15:59:11.677175 27515 catalog_manager.cc:4841] TS 3a20f42ff328424781a4018d5a9064cb (127.25.31.131:38759): tablet ae251ba45c1a4642820db5f455ce6c35 (table test-table [id=154b3709af6149c1b788b308ea6946e0]) successfully deleted
I20260806 15:59:11.677363 28152 ts_tablet_manager.cc:1918] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 15:59:11.677450 28152 log.cc:1196] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/ae251ba45c1a4642820db5f455ce6c35
I20260806 15:59:11.677743 28152 ts_tablet_manager.cc:1939] T ae251ba45c1a4642820db5f455ce6c35 P d872c956af834b7c8d8e1237af06f06c: Deleting consensus metadata
I20260806 15:59:11.678473 27517 catalog_manager.cc:4841] TS d872c956af834b7c8d8e1237af06f06c (127.25.31.130:39883): tablet ae251ba45c1a4642820db5f455ce6c35 (table test-table [id=154b3709af6149c1b788b308ea6946e0]) successfully deleted
Aug 06 15:59:11 dist-test-slave-tm1g krb5kdc[27485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031951, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 15:59:11 dist-test-slave-tm1g krb5kdc[27485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031951, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:11.700672 28161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:11.691968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42884 (local address 127.25.31.190:42679)
0806 15:59:11.692254 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:11.692264 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:11.692355 (+ 91us) server_negotiation.cc:407] Connection header received
0806 15:59:11.692572 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:11.692576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:11.692636 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:11.692741 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:11.693444 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:11.694016 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:11.694920 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:11.695162 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:11.695272 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:11.697636 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:11.697639 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:11.697641 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:11.697664 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:11.699249 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:11.699626 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:11.699629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:11.699630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:11.699704 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:11.699964 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:11.699966 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:11.699967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:11.700144 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:11.700246 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:11.700451 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 15:59:11.700548 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":95,"threads_started":1}
I20260806 15:59:11.703150 27532 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:59:11.703590 27532 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:11.708230 27673 tablet_service.cc:1467] Processing CreateTablet for tablet cb767350593444ebaba2ab4790ccbf11 (DEFAULT_TABLE table=test-table [id=4f5e23ac8a394156baee746bdd92a50b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:11.708281 27820 tablet_service.cc:1467] Processing CreateTablet for tablet cb767350593444ebaba2ab4790ccbf11 (DEFAULT_TABLE table=test-table [id=4f5e23ac8a394156baee746bdd92a50b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:11.708379 27967 tablet_service.cc:1467] Processing CreateTablet for tablet cb767350593444ebaba2ab4790ccbf11 (DEFAULT_TABLE table=test-table [id=4f5e23ac8a394156baee746bdd92a50b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:11.708552 27820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb767350593444ebaba2ab4790ccbf11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:11.708551 27673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb767350593444ebaba2ab4790ccbf11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:11.708679 27967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb767350593444ebaba2ab4790ccbf11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:11.711113 28164 tablet_bootstrap.cc:492] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c: Bootstrap starting.
I20260806 15:59:11.711244 28163 tablet_bootstrap.cc:492] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0: Bootstrap starting.
I20260806 15:59:11.712049 28164 tablet_bootstrap.cc:654] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:11.712049 28163 tablet_bootstrap.cc:654] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:11.712692 28162 tablet_bootstrap.cc:492] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb: Bootstrap starting.
I20260806 15:59:11.713388 28163 tablet_bootstrap.cc:492] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0: No bootstrap required, opened a new log
I20260806 15:59:11.713481 28163 ts_tablet_manager.cc:1397] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:59:11.713515 28164 tablet_bootstrap.cc:492] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c: No bootstrap required, opened a new log
I20260806 15:59:11.713567 28162 tablet_bootstrap.cc:654] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:11.713584 28164 ts_tablet_manager.cc:1397] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 15:59:11.713904 28163 raft_consensus.cc:348] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.713924 28164 raft_consensus.cc:348] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.714031 28164 raft_consensus.cc:374] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:11.714031 28163 raft_consensus.cc:374] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:11.714073 28163 raft_consensus.cc:729] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f76b7d35635404b95797b18318d42c0, State: Initialized, Role: FOLLOWER
I20260806 15:59:11.714073 28164 raft_consensus.cc:729] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d872c956af834b7c8d8e1237af06f06c, State: Initialized, Role: FOLLOWER
I20260806 15:59:11.714205 28164 consensus_queue.cc:260] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.714257 28163 consensus_queue.cc:260] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.714532 28164 ts_tablet_manager.cc:1428] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:59:11.714653 28163 ts_tablet_manager.cc:1428] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:59:11.715204 28162 tablet_bootstrap.cc:492] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb: No bootstrap required, opened a new log
I20260806 15:59:11.715294 28162 ts_tablet_manager.cc:1397] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 15:59:11.715721 28162 raft_consensus.cc:348] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.715843 28162 raft_consensus.cc:374] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:11.715878 28162 raft_consensus.cc:729] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a20f42ff328424781a4018d5a9064cb, State: Initialized, Role: FOLLOWER
I20260806 15:59:11.716043 28162 consensus_queue.cc:260] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.716467 28162 ts_tablet_manager.cc:1428] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:59:11.744590 28146 raft_consensus.cc:482] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:11.744719 28146 raft_consensus.cc:504] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.745134 28146 leader_election.cc:288] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f76b7d35635404b95797b18318d42c0 (127.25.31.129:39385), 3a20f42ff328424781a4018d5a9064cb (127.25.31.131:38759)
I20260806 15:59:11.745671 27693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb767350593444ebaba2ab4790ccbf11" candidate_uuid: "d872c956af834b7c8d8e1237af06f06c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f76b7d35635404b95797b18318d42c0" is_pre_election: true
I20260806 15:59:11.745832 27693 raft_consensus.cc:2454] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d872c956af834b7c8d8e1237af06f06c in term 0.
I20260806 15:59:11.746171 27767 leader_election.cc:302] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f76b7d35635404b95797b18318d42c0, d872c956af834b7c8d8e1237af06f06c; no voters:
I20260806 15:59:11.746418 27987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb767350593444ebaba2ab4790ccbf11" candidate_uuid: "d872c956af834b7c8d8e1237af06f06c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a20f42ff328424781a4018d5a9064cb" is_pre_election: true
I20260806 15:59:11.746596 28146 raft_consensus.cc:2781] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:11.746584 27987 raft_consensus.cc:2454] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d872c956af834b7c8d8e1237af06f06c in term 0.
I20260806 15:59:11.746680 28146 raft_consensus.cc:482] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:11.746757 28146 raft_consensus.cc:3037] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:11.748286 28146 raft_consensus.cc:504] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.748664 28146 leader_election.cc:288] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [CANDIDATE]: Term 1 election: Requested vote from peers 9f76b7d35635404b95797b18318d42c0 (127.25.31.129:39385), 3a20f42ff328424781a4018d5a9064cb (127.25.31.131:38759)
I20260806 15:59:11.749068 27987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb767350593444ebaba2ab4790ccbf11" candidate_uuid: "d872c956af834b7c8d8e1237af06f06c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a20f42ff328424781a4018d5a9064cb"
I20260806 15:59:11.749071 27693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb767350593444ebaba2ab4790ccbf11" candidate_uuid: "d872c956af834b7c8d8e1237af06f06c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f76b7d35635404b95797b18318d42c0"
I20260806 15:59:11.749161 27987 raft_consensus.cc:3037] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:11.749217 27693 raft_consensus.cc:3037] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:11.750366 27987 raft_consensus.cc:2454] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d872c956af834b7c8d8e1237af06f06c in term 1.
I20260806 15:59:11.750402 27693 raft_consensus.cc:2454] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d872c956af834b7c8d8e1237af06f06c in term 1.
I20260806 15:59:11.750663 27767 leader_election.cc:302] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f76b7d35635404b95797b18318d42c0, d872c956af834b7c8d8e1237af06f06c; no voters:
I20260806 15:59:11.750825 28146 raft_consensus.cc:2781] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:11.750911 28146 raft_consensus.cc:686] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [term 1 LEADER]: Becoming Leader. State: Replica: d872c956af834b7c8d8e1237af06f06c, State: Running, Role: LEADER
I20260806 15:59:11.751044 28146 consensus_queue.cc:237] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.752669 27530 catalog_manager.cc:5495] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c reported cstate change: term changed from 0 to 1, leader changed from <none> to d872c956af834b7c8d8e1237af06f06c (127.25.31.130). New cstate: current_term: 1 leader_uuid: "d872c956af834b7c8d8e1237af06f06c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } health_report { overall_health: UNKNOWN } } }
I20260806 15:59:11.771543 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:11.768117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53270 (local address 127.25.31.130:39883)
0806 15:59:11.768391 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:11.768400 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:11.768412 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:59:11.768456 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:11.768459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:11.768497 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:11.768587 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:11.768857 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:11.769400 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:11.770415 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:11.770660 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:11.770773 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:11.771034 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:11.771163 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:11.771245 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:11.771337 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 15:59:11.771390 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":98,"threads_started":1}
I20260806 15:59:11.777091 27988 raft_consensus.cc:1261] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Refusing update from remote peer d872c956af834b7c8d8e1237af06f06c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:11.777102 27693 raft_consensus.cc:1261] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Refusing update from remote peer d872c956af834b7c8d8e1237af06f06c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:11.777455 28146 consensus_queue.cc:1035] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:11.777590 28142 consensus_queue.cc:1035] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:11.803217 27533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:11.803363 27533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:11.805085 27533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e: Sending DeleteTablet for 3 replicas of tablet cb767350593444ebaba2ab4790ccbf11
I20260806 15:59:11.805570 27673 tablet_service.cc:1514] Processing DeleteTablet for tablet cb767350593444ebaba2ab4790ccbf11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:11 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:33634
I20260806 15:59:11.805605 27820 tablet_service.cc:1514] Processing DeleteTablet for tablet cb767350593444ebaba2ab4790ccbf11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:11 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:53252
I20260806 15:59:11.805681 27967 tablet_service.cc:1514] Processing DeleteTablet for tablet cb767350593444ebaba2ab4790ccbf11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:11 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:45074
I20260806 15:59:11.805754 28152 tablet_replica.cc:330] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c: stopping tablet replica
I20260806 15:59:11.805760 28151 tablet_replica.cc:330] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0: stopping tablet replica
I20260806 15:59:11.805841 28153 tablet_replica.cc:330] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb: stopping tablet replica
I20260806 15:59:11.805858 28152 raft_consensus.cc:2229] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:59:11.805858 28151 raft_consensus.cc:2229] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:11.805963 28151 raft_consensus.cc:2258] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:11.806038 28152 raft_consensus.cc:2258] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:11.806263 28153 raft_consensus.cc:2229] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:11.806421 28153 raft_consensus.cc:2258] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:11.806766 28151 ts_tablet_manager.cc:1905] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:11.806977 28152 ts_tablet_manager.cc:1905] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:11.807035 27533 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 15:59:11.807113 28153 ts_tablet_manager.cc:1905] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 15:59:11.807403 27533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:11.809350 28152 ts_tablet_manager.cc:1918] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:59:11.809402 28152 log.cc:1196] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/cb767350593444ebaba2ab4790ccbf11
I20260806 15:59:11.809391 28151 ts_tablet_manager.cc:1918] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:59:11.809470 28151 log.cc:1196] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-0/wal/wals/cb767350593444ebaba2ab4790ccbf11
I20260806 15:59:11.809600 28152 ts_tablet_manager.cc:1939] T cb767350593444ebaba2ab4790ccbf11 P d872c956af834b7c8d8e1237af06f06c: Deleting consensus metadata
I20260806 15:59:11.809634 28153 ts_tablet_manager.cc:1918] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:59:11.809706 28153 log.cc:1196] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/cb767350593444ebaba2ab4790ccbf11
I20260806 15:59:11.809736 28151 ts_tablet_manager.cc:1939] T cb767350593444ebaba2ab4790ccbf11 P 9f76b7d35635404b95797b18318d42c0: Deleting consensus metadata
I20260806 15:59:11.809979 28153 ts_tablet_manager.cc:1939] T cb767350593444ebaba2ab4790ccbf11 P 3a20f42ff328424781a4018d5a9064cb: Deleting consensus metadata
I20260806 15:59:11.810346 27517 catalog_manager.cc:4841] TS d872c956af834b7c8d8e1237af06f06c (127.25.31.130:39883): tablet cb767350593444ebaba2ab4790ccbf11 (table test-table [id=4f5e23ac8a394156baee746bdd92a50b]) successfully deleted
I20260806 15:59:11.810473 27515 catalog_manager.cc:4841] TS 9f76b7d35635404b95797b18318d42c0 (127.25.31.129:39385): tablet cb767350593444ebaba2ab4790ccbf11 (table test-table [id=4f5e23ac8a394156baee746bdd92a50b]) successfully deleted
I20260806 15:59:11.811082 27515 catalog_manager.cc:4841] TS 3a20f42ff328424781a4018d5a9064cb (127.25.31.131:38759): tablet cb767350593444ebaba2ab4790ccbf11 (table test-table [id=4f5e23ac8a394156baee746bdd92a50b]) successfully deleted
I20260806 15:59:11.813104 27673 tablet_service.cc:1467] Processing CreateTablet for tablet 0c65f6315e0c4753a5249df1d63b1426 (DEFAULT_TABLE table=test-table [id=02a10c0e6ed744de93849e667d82d310]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:11.813177 27820 tablet_service.cc:1467] Processing CreateTablet for tablet 0c65f6315e0c4753a5249df1d63b1426 (DEFAULT_TABLE table=test-table [id=02a10c0e6ed744de93849e667d82d310]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:11.813359 27673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c65f6315e0c4753a5249df1d63b1426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:11.813325 27967 tablet_service.cc:1467] Processing CreateTablet for tablet 0c65f6315e0c4753a5249df1d63b1426 (DEFAULT_TABLE table=test-table [id=02a10c0e6ed744de93849e667d82d310]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:11.813419 27820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c65f6315e0c4753a5249df1d63b1426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:11.813540 27967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c65f6315e0c4753a5249df1d63b1426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:11.815462 28163 tablet_bootstrap.cc:492] T 0c65f6315e0c4753a5249df1d63b1426 P 9f76b7d35635404b95797b18318d42c0: Bootstrap starting.
I20260806 15:59:11.815645 28162 tablet_bootstrap.cc:492] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb: Bootstrap starting.
I20260806 15:59:11.815645 28164 tablet_bootstrap.cc:492] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c: Bootstrap starting.
I20260806 15:59:11.816205 28163 tablet_bootstrap.cc:654] T 0c65f6315e0c4753a5249df1d63b1426 P 9f76b7d35635404b95797b18318d42c0: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:11.816499 28164 tablet_bootstrap.cc:654] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:11.816499 28162 tablet_bootstrap.cc:654] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:11.817706 28163 tablet_bootstrap.cc:492] T 0c65f6315e0c4753a5249df1d63b1426 P 9f76b7d35635404b95797b18318d42c0: No bootstrap required, opened a new log
I20260806 15:59:11.817800 28163 ts_tablet_manager.cc:1397] T 0c65f6315e0c4753a5249df1d63b1426 P 9f76b7d35635404b95797b18318d42c0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:59:11.817813 28164 tablet_bootstrap.cc:492] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c: No bootstrap required, opened a new log
I20260806 15:59:11.817884 28164 ts_tablet_manager.cc:1397] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:59:11.818058 28162 tablet_bootstrap.cc:492] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb: No bootstrap required, opened a new log
I20260806 15:59:11.818161 28162 ts_tablet_manager.cc:1397] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 15:59:11.818226 28163 raft_consensus.cc:348] T 0c65f6315e0c4753a5249df1d63b1426 P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.818226 28164 raft_consensus.cc:348] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.818367 28163 raft_consensus.cc:374] T 0c65f6315e0c4753a5249df1d63b1426 P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:11.818367 28164 raft_consensus.cc:374] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:11.818408 28163 raft_consensus.cc:729] T 0c65f6315e0c4753a5249df1d63b1426 P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f76b7d35635404b95797b18318d42c0, State: Initialized, Role: FOLLOWER
I20260806 15:59:11.818408 28164 raft_consensus.cc:729] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d872c956af834b7c8d8e1237af06f06c, State: Initialized, Role: FOLLOWER
I20260806 15:59:11.818534 28163 consensus_queue.cc:260] T 0c65f6315e0c4753a5249df1d63b1426 P 9f76b7d35635404b95797b18318d42c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.818534 28164 consensus_queue.cc:260] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.818635 28162 raft_consensus.cc:348] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.818742 28162 raft_consensus.cc:374] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:11.818785 28162 raft_consensus.cc:729] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a20f42ff328424781a4018d5a9064cb, State: Initialized, Role: FOLLOWER
I20260806 15:59:11.818796 28163 ts_tablet_manager.cc:1428] T 0c65f6315e0c4753a5249df1d63b1426 P 9f76b7d35635404b95797b18318d42c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:59:11.818796 28164 ts_tablet_manager.cc:1428] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:59:11.818913 28162 consensus_queue.cc:260] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.819169 28162 ts_tablet_manager.cc:1428] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:59:11.939824 28168 raft_consensus.cc:482] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:11.939989 28168 raft_consensus.cc:504] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.940497 28168 leader_election.cc:288] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f76b7d35635404b95797b18318d42c0 (127.25.31.129:39385), d872c956af834b7c8d8e1237af06f06c (127.25.31.130:39883)
I20260806 15:59:11.941022 27839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c65f6315e0c4753a5249df1d63b1426" candidate_uuid: "3a20f42ff328424781a4018d5a9064cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d872c956af834b7c8d8e1237af06f06c" is_pre_election: true
I20260806 15:59:11.941020 27693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c65f6315e0c4753a5249df1d63b1426" candidate_uuid: "3a20f42ff328424781a4018d5a9064cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f76b7d35635404b95797b18318d42c0" is_pre_election: true
I20260806 15:59:11.941190 27839 raft_consensus.cc:2454] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a20f42ff328424781a4018d5a9064cb in term 0.
I20260806 15:59:11.941190 27693 raft_consensus.cc:2454] T 0c65f6315e0c4753a5249df1d63b1426 P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a20f42ff328424781a4018d5a9064cb in term 0.
I20260806 15:59:11.941543 27917 leader_election.cc:302] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a20f42ff328424781a4018d5a9064cb, d872c956af834b7c8d8e1237af06f06c; no voters:
I20260806 15:59:11.941748 28168 raft_consensus.cc:2781] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:11.941808 28168 raft_consensus.cc:482] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:11.941835 28168 raft_consensus.cc:3037] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:11.943005 28168 raft_consensus.cc:504] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.943328 28168 leader_election.cc:288] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [CANDIDATE]: Term 1 election: Requested vote from peers 9f76b7d35635404b95797b18318d42c0 (127.25.31.129:39385), d872c956af834b7c8d8e1237af06f06c (127.25.31.130:39883)
I20260806 15:59:11.943697 27693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c65f6315e0c4753a5249df1d63b1426" candidate_uuid: "3a20f42ff328424781a4018d5a9064cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f76b7d35635404b95797b18318d42c0"
I20260806 15:59:11.943698 27839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c65f6315e0c4753a5249df1d63b1426" candidate_uuid: "3a20f42ff328424781a4018d5a9064cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d872c956af834b7c8d8e1237af06f06c"
I20260806 15:59:11.943841 27839 raft_consensus.cc:3037] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:11.943837 27693 raft_consensus.cc:3037] T 0c65f6315e0c4753a5249df1d63b1426 P 9f76b7d35635404b95797b18318d42c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:11.945029 27693 raft_consensus.cc:2454] T 0c65f6315e0c4753a5249df1d63b1426 P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a20f42ff328424781a4018d5a9064cb in term 1.
I20260806 15:59:11.945030 27839 raft_consensus.cc:2454] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a20f42ff328424781a4018d5a9064cb in term 1.
I20260806 15:59:11.945336 27916 leader_election.cc:302] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a20f42ff328424781a4018d5a9064cb, 9f76b7d35635404b95797b18318d42c0; no voters:
I20260806 15:59:11.945539 28168 raft_consensus.cc:2781] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:11.945765 28168 raft_consensus.cc:686] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [term 1 LEADER]: Becoming Leader. State: Replica: 3a20f42ff328424781a4018d5a9064cb, State: Running, Role: LEADER
I20260806 15:59:11.945922 28168 consensus_queue.cc:237] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } }
I20260806 15:59:11.947570 27530 catalog_manager.cc:5495] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a20f42ff328424781a4018d5a9064cb (127.25.31.131). New cstate: current_term: 1 leader_uuid: "3a20f42ff328424781a4018d5a9064cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a20f42ff328424781a4018d5a9064cb" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38759 } health_report { overall_health: HEALTHY } } }
I20260806 15:59:11.974637 28181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:11.971610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45088 (local address 127.25.31.131:38759)
0806 15:59:11.971896 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:11.971903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:11.971914 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:59:11.971952 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:11.971954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:11.971992 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:11.972069 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:11.972286 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:11.972826 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:11.973593 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:11.973820 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:11.973894 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:11.974113 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:11.974180 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:11.974234 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:11.974360 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 15:59:11.974403 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":127,"threads_started":1}
I20260806 15:59:11.980656 27693 raft_consensus.cc:1261] T 0c65f6315e0c4753a5249df1d63b1426 P 9f76b7d35635404b95797b18318d42c0 [term 1 FOLLOWER]: Refusing update from remote peer 3a20f42ff328424781a4018d5a9064cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:11.980683 27839 raft_consensus.cc:1261] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Refusing update from remote peer 3a20f42ff328424781a4018d5a9064cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:11.981025 28168 consensus_queue.cc:1035] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d872c956af834b7c8d8e1237af06f06c" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:11.981166 28179 consensus_queue.cc:1035] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f76b7d35635404b95797b18318d42c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 39385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 15:59:11.994095 28133 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 15:59:13.542752 27532 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:13.542927 27532 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:13.545240 27532 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c0e5acbab6045b8b6ec81c1bd02416e: Sending DeleteTablet for 3 replicas of tablet 0c65f6315e0c4753a5249df1d63b1426
I20260806 15:59:13.545739 27820 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c65f6315e0c4753a5249df1d63b1426 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:13 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:53252
I20260806 15:59:13.545836 27967 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c65f6315e0c4753a5249df1d63b1426 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:13 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:45074
I20260806 15:59:13.546154 28211 tablet_replica.cc:330] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb: stopping tablet replica
I20260806 15:59:13.546124 27673 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c65f6315e0c4753a5249df1d63b1426 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:13 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:33634
I20260806 15:59:13.546147 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 27606
I20260806 15:59:13.547216 28211 raft_consensus.cc:2229] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:59:13.547281 28210 tablet_replica.cc:330] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c: stopping tablet replica
I20260806 15:59:13.547433 28210 raft_consensus.cc:2229] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:13.547426 28211 raft_consensus.cc:2258] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:13.547549 28210 raft_consensus.cc:2258] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:13.548240 28210 ts_tablet_manager.cc:1905] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:13.548306 28211 ts_tablet_manager.cc:1905] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:13.550549 28211 ts_tablet_manager.cc:1918] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 15:59:13.550642 28211 log.cc:1196] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/0c65f6315e0c4753a5249df1d63b1426
I20260806 15:59:13.550915 28211 ts_tablet_manager.cc:1939] T 0c65f6315e0c4753a5249df1d63b1426 P 3a20f42ff328424781a4018d5a9064cb: Deleting consensus metadata
I20260806 15:59:13.551628 27515 catalog_manager.cc:4841] TS 3a20f42ff328424781a4018d5a9064cb (127.25.31.131:38759): tablet 0c65f6315e0c4753a5249df1d63b1426 (table test-table [id=02a10c0e6ed744de93849e667d82d310]) successfully deleted
I20260806 15:59:13.551698 28210 ts_tablet_manager.cc:1918] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 15:59:13.551781 28210 log.cc:1196] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/0c65f6315e0c4753a5249df1d63b1426
I20260806 15:59:13.552062 28210 ts_tablet_manager.cc:1939] T 0c65f6315e0c4753a5249df1d63b1426 P d872c956af834b7c8d8e1237af06f06c: Deleting consensus metadata
I20260806 15:59:13.552845 27517 catalog_manager.cc:4841] TS d872c956af834b7c8d8e1237af06f06c (127.25.31.130:39883): tablet 0c65f6315e0c4753a5249df1d63b1426 (table test-table [id=02a10c0e6ed744de93849e667d82d310]) successfully deleted
I20260806 15:59:13.556011 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 27751
W20260806 15:59:13.556028 27515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:39385 (error 108)
I20260806 15:59:13.557443 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:13.557034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:39385 (local address 127.0.0.1:33644)
0806 15:59:13.557291 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:13.557363 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:39385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
W20260806 15:59:13.557546 27515 catalog_manager.cc:4584] TS 9f76b7d35635404b95797b18318d42c0 (127.25.31.129:39385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0c65f6315e0c4753a5249df1d63b1426: Network error: Client connection negotiation failed: client connection to 127.25.31.129:39385: connect: Connection refused (error 111)
I20260806 15:59:13.565038 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 27898
I20260806 15:59:13.572321 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 27499
2026-08-06T15:59:13Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6398 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:59:13 dist-test-slave-tm1g krb5kdc[28216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:59:13 dist-test-slave-tm1g krb5kdc[28216](info): set up 2 sockets
Aug 06 15:59:13 dist-test-slave-tm1g krb5kdc[28216](info): commencing operation
krb5kdc: starting...
W20260806 15:59:14.639521 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 15:59:14 dist-test-slave-tm1g krb5kdc[28216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:59:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 15:59:14.755116 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:38419
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:42607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:38419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:14.863566 28230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:14.863798 28230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:14.863837 28230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:14.867758 28230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:14.867810 28230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:14.867828 28230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:14.867885 28230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:14.867910 28230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:14.873064 28230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:42607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:38419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:38419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:14.874013 28230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:14.875013 28230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:14.883646 28236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:14.883637 28235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:14.883725 28230 server_base.cc:1034] running on GCE node
W20260806 15:59:14.883639 28238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:14.884375 28230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:14.885390 28230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:14.886608 28230 hybrid_clock.cc:648] HybridClock initialized: now 1786031954886582 us; error 43 us; skew 500 ppm
Aug 06 15:59:14 dist-test-slave-tm1g krb5kdc[28216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031954, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:14.889432 28230 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 15:59:14.890375 28230 webserver.cc:460] Webserver started at http://127.25.31.190:34323/ using document root <none> and password file <none>
I20260806 15:59:14.890923 28230 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:14.891016 28230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:14.891244 28230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:14.893209 28230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "620b9c8182494ff2b3a6b8887707e977"
format_stamp: "Formatted at 2026-08-06 15:59:14 on dist-test-slave-tm1g"
server_key: "8ebe04279b1f24a3867ff15d668bcda0"
server_key_iv: "d3df9d0335d77a02b960d06549bf8aaf"
server_key_version: "encryptionkey@0"
I20260806 15:59:14.893782 28230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "620b9c8182494ff2b3a6b8887707e977"
format_stamp: "Formatted at 2026-08-06 15:59:14 on dist-test-slave-tm1g"
server_key: "8ebe04279b1f24a3867ff15d668bcda0"
server_key_iv: "d3df9d0335d77a02b960d06549bf8aaf"
server_key_version: "encryptionkey@0"
I20260806 15:59:14.897404 28230 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 15:59:14.899833 28245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:14.900983 28230 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 15:59:14.901109 28230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "620b9c8182494ff2b3a6b8887707e977"
format_stamp: "Formatted at 2026-08-06 15:59:14 on dist-test-slave-tm1g"
server_key: "8ebe04279b1f24a3867ff15d668bcda0"
server_key_iv: "d3df9d0335d77a02b960d06549bf8aaf"
server_key_version: "encryptionkey@0"
I20260806 15:59:14.901211 28230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:14.919276 28230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:14.922518 28230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:14.922716 28230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:14.929946 28230 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:38419
I20260806 15:59:14.929970 28301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:38419 every 8 connection(s)
I20260806 15:59:14.930935 28230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:14.933475 28302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:14.938696 28302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977: Bootstrap starting.
I20260806 15:59:14.940659 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 28230
I20260806 15:59:14.940749 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:14.941022 25726 external_mini_cluster.cc:1428] Setting key a4942e0db1350e89ac55db774ca1e78a
I20260806 15:59:14.941112 28302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:14.942041 28302 log.cc:824] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:14.944069 28302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977: No bootstrap required, opened a new log
I20260806 15:59:14.948170 28302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620b9c8182494ff2b3a6b8887707e977" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 38419 } }
I20260806 15:59:14.948438 28302 raft_consensus.cc:374] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:14.948493 28302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 620b9c8182494ff2b3a6b8887707e977, State: Initialized, Role: FOLLOWER
I20260806 15:59:14.949018 28302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620b9c8182494ff2b3a6b8887707e977" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 38419 } }
I20260806 15:59:14.949204 28302 raft_consensus.cc:388] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:14.949270 28302 raft_consensus.cc:482] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:14.949368 28302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 15:59:14 dist-test-slave-tm1g krb5kdc[28216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:14.950373 28302 raft_consensus.cc:504] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620b9c8182494ff2b3a6b8887707e977" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 38419 } }
I20260806 15:59:14.950702 28302 leader_election.cc:302] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 620b9c8182494ff2b3a6b8887707e977; no voters:
I20260806 15:59:14.951052 28302 leader_election.cc:288] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:14.951205 28308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:14.951439 28308 raft_consensus.cc:686] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [term 1 LEADER]: Becoming Leader. State: Replica: 620b9c8182494ff2b3a6b8887707e977, State: Running, Role: LEADER
I20260806 15:59:14.951783 28308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620b9c8182494ff2b3a6b8887707e977" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 38419 } }
I20260806 15:59:14.952288 28302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:14.953446 28309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "620b9c8182494ff2b3a6b8887707e977" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620b9c8182494ff2b3a6b8887707e977" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 38419 } } }
I20260806 15:59:14.953567 28309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:14.953961 28317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:14.953900 28310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 620b9c8182494ff2b3a6b8887707e977. Latest consensus state: current_term: 1 leader_uuid: "620b9c8182494ff2b3a6b8887707e977" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "620b9c8182494ff2b3a6b8887707e977" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 38419 } } }
I20260806 15:59:14.954069 28310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:14.956267 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:14.942128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36638 (local address 127.25.31.190:38419)
0806 15:59:14.942649 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:14.942670 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:14.942715 (+ 45us) server_negotiation.cc:407] Connection header received
0806 15:59:14.943399 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:14.943415 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:14.943718 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:14.943990 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:14.944906 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:14.945916 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:14.947245 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:14.947708 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:14.947830 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:14.950640 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:14.950646 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:14.950658 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:14.950683 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:14.952783 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:14.953396 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:14.953403 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:14.953409 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:14.953589 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:14.953920 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:14.953923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:14.953925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:14.954472 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:14.954660 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:14.955059 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 15:59:14.955354 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":202,"threads_started":1}
I20260806 15:59:14.957620 28317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:14.962919 28317 catalog_manager.cc:1347] Generated new cluster ID: b688ec5dc87848dd934fbe5229a02603
I20260806 15:59:14.962978 28317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:14.980054 28317 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:14.980971 28317 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:14.988644 28317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 620b9c8182494ff2b3a6b8887707e977: Generated new TSK 0
I20260806 15:59:14.989275 28317 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 15:59:15.024746 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:38419
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:42607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:15.134521 28330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:15.134753 28330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:15.134793 28330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:15.138706 28330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:15.138754 28330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:15.138845 28330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:15.143934 28330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:42607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:38419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:15.144893 28330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:15.145887 28330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:15.155396 28336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:15.155470 28330 server_base.cc:1034] running on GCE node
W20260806 15:59:15.155402 28335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:15.155405 28338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:15.156023 28330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:15.156641 28330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:15.157835 28330 hybrid_clock.cc:648] HybridClock initialized: now 1786031955157807 us; error 43 us; skew 500 ppm
Aug 06 15:59:15 dist-test-slave-tm1g krb5kdc[28216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031955, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:15.160817 28330 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 15:59:15.161639 28330 webserver.cc:460] Webserver started at http://127.25.31.129:45305/ using document root <none> and password file <none>
I20260806 15:59:15.162189 28330 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:15.162250 28330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:15.162484 28330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:15.164463 28330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "4b96b3a736a84f51b2b3e9c325fc661f"
format_stamp: "Formatted at 2026-08-06 15:59:15 on dist-test-slave-tm1g"
server_key: "478264e37a2c455673d1aa946b9ab6e5"
server_key_iv: "46a2edab961fb874db13b4aaabb3ceb5"
server_key_version: "encryptionkey@0"
I20260806 15:59:15.165017 28330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "4b96b3a736a84f51b2b3e9c325fc661f"
format_stamp: "Formatted at 2026-08-06 15:59:15 on dist-test-slave-tm1g"
server_key: "478264e37a2c455673d1aa946b9ab6e5"
server_key_iv: "46a2edab961fb874db13b4aaabb3ceb5"
server_key_version: "encryptionkey@0"
I20260806 15:59:15.168689 28330 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 15:59:15.171100 28346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:15.172251 28330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 15:59:15.172374 28330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "4b96b3a736a84f51b2b3e9c325fc661f"
format_stamp: "Formatted at 2026-08-06 15:59:15 on dist-test-slave-tm1g"
server_key: "478264e37a2c455673d1aa946b9ab6e5"
server_key_iv: "46a2edab961fb874db13b4aaabb3ceb5"
server_key_version: "encryptionkey@0"
I20260806 15:59:15.172475 28330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:15.202934 28330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:15.206501 28330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:15.206709 28330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:15.207315 28330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:15.208128 28330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:15.208178 28330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:15.208217 28330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:15.208259 28330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:15.217460 28330 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:39125
I20260806 15:59:15.217494 28467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:39125 every 8 connection(s)
I20260806 15:59:15.218350 28330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:15.221257 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 28330
I20260806 15:59:15.221400 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 15:59:15.221752 25726 external_mini_cluster.cc:1428] Setting key 6da84ec950066f7c59fb80be41b09ccf
Aug 06 15:59:15 dist-test-slave-tm1g krb5kdc[28216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031955, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
I20260806 15:59:15.233237 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:15.220074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:60215 (local address 127.25.31.190:38419)
0806 15:59:15.220207 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:15.220211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:15.221098 (+ 887us) server_negotiation.cc:407] Connection header received
0806 15:59:15.222751 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:15.222755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:15.222810 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:15.222898 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:15.224397 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:15.224980 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:15.226150 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:15.226477 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:15.226592 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:15.229465 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:15.229468 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:15.229470 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:15.229492 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:15.230983 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:15.231607 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:15.231609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:15.231611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:15.231686 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:15.232130 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:15.232133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:15.232134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:15.232305 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:15.232386 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:15.232875 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 15:59:15.233024 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 15:59:15.233934 28470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:15.220435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:38419 (local address 127.25.31.129:60215)
0806 15:59:15.220911 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:15.220953 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:15.222548 (+ 1595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:15.223081 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:15.223088 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:15.223535 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:15.224229 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:15.224252 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:15.225123 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:15.225126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:15.226019 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:15.226029 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:15.226633 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:15.226637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:15.226800 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:15.227411 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:15.227426 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:15.229292 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:15.231108 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:15.231116 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:15.231131 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:15.231505 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:15.231776 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:15.231779 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:15.231781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:15.231976 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:15.232410 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:15.232415 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:15.232628 (+ 213us) client_negotiation.cc:769] Sending connection context
0806 15:59:15.232781 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 15:59:15.233104 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":120,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
I20260806 15:59:15.235392 28468 heartbeater.cc:347] Connected to a master server at 127.25.31.190:38419
I20260806 15:59:15.235657 28468 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:15.236153 28468 heartbeater.cc:511] Master 127.25.31.190:38419 requested a full tablet report, sending...
I20260806 15:59:15.237805 28262 ts_manager.cc:194] Registered new tserver with Master: 4b96b3a736a84f51b2b3e9c325fc661f (127.25.31.129:39125)
I20260806 15:59:15.238968 28262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:60215
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 15:59:15.251260 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:38419
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:42607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:15.359612 28473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:15.359839 28473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:15.359879 28473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:15.363725 28473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:15.363773 28473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:15.363827 28473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:15.368907 28473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:42607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:38419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:15.369803 28473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:15.370748 28473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:15.380023 28479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:15.379979 28481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:15.380129 28473 server_base.cc:1034] running on GCE node
W20260806 15:59:15.380156 28478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:15.380630 28473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:15.381237 28473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:15.382398 28473 hybrid_clock.cc:648] HybridClock initialized: now 1786031955382394 us; error 33 us; skew 500 ppm
Aug 06 15:59:15 dist-test-slave-tm1g krb5kdc[28216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031955, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:15.385003 28473 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 15:59:15.385824 28473 webserver.cc:460] Webserver started at http://127.25.31.130:38091/ using document root <none> and password file <none>
I20260806 15:59:15.386430 28473 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:15.386520 28473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:15.386750 28473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:15.388698 28473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "6d07ad7432bd465ba2c51676f57f03bb"
format_stamp: "Formatted at 2026-08-06 15:59:15 on dist-test-slave-tm1g"
server_key: "a4e141e5e9294947e8d87a50a414cabe"
server_key_iv: "eb367c8e9fe39d59e02e0d07294e312f"
server_key_version: "encryptionkey@0"
I20260806 15:59:15.389257 28473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "6d07ad7432bd465ba2c51676f57f03bb"
format_stamp: "Formatted at 2026-08-06 15:59:15 on dist-test-slave-tm1g"
server_key: "a4e141e5e9294947e8d87a50a414cabe"
server_key_iv: "eb367c8e9fe39d59e02e0d07294e312f"
server_key_version: "encryptionkey@0"
I20260806 15:59:15.392879 28473 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 15:59:15.395417 28490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:15.396675 28473 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:15.396816 28473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "6d07ad7432bd465ba2c51676f57f03bb"
format_stamp: "Formatted at 2026-08-06 15:59:15 on dist-test-slave-tm1g"
server_key: "a4e141e5e9294947e8d87a50a414cabe"
server_key_iv: "eb367c8e9fe39d59e02e0d07294e312f"
server_key_version: "encryptionkey@0"
I20260806 15:59:15.396929 28473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:15.405827 28473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:15.408699 28473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:15.408902 28473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:15.409459 28473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:15.410248 28473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:15.410374 28473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:15.410454 28473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:15.410511 28473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:15.419929 28473 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:37597
I20260806 15:59:15.419955 28608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:37597 every 8 connection(s)
I20260806 15:59:15.420883 28473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:15.427023 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 28473
I20260806 15:59:15.427135 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 15:59:15.427451 25726 external_mini_cluster.cc:1428] Setting key 8ecb6bcfc303636dc2f2507a8e3ee094
Aug 06 15:59:15 dist-test-slave-tm1g krb5kdc[28216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031955, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:15.435352 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:15.422645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:41051 (local address 127.25.31.190:38419)
0806 15:59:15.422800 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:15.422803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:15.423681 (+ 878us) server_negotiation.cc:407] Connection header received
0806 15:59:15.425047 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:15.425050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:15.425107 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:15.425215 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:15.426448 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:15.427203 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:15.428647 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:15.428897 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:15.429014 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:15.431518 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:15.431522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:15.431527 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:15.431555 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:15.432932 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:15.433584 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:15.433587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:15.433588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:15.433697 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:15.434066 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:15.434071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:15.434076 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:15.434391 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:15.434503 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:15.435010 (+ 507us) server_negotiation.cc:299] Negotiation successful
0806 15:59:15.435157 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 15:59:15.436071 28611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:15.423034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:38419 (local address 127.25.31.130:41051)
0806 15:59:15.423517 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:15.423561 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:15.424834 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:15.425399 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:15.425406 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:15.425765 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:15.426232 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:15.426244 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:15.427360 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:15.427363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:15.428515 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:15.428526 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:15.429053 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:15.429057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:15.429231 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:15.429859 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:15.429875 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:15.431351 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:15.433070 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:15.433079 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:15.433097 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:15.433478 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:15.433804 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:15.433806 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:15.433808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:15.433963 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:15.434527 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:15.434532 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:15.434760 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 15:59:15.434920 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 15:59:15.435183 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":117,"threads_started":1}
I20260806 15:59:15.437268 28609 heartbeater.cc:347] Connected to a master server at 127.25.31.190:38419
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
I20260806 15:59:15.437532 28609 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:15.438019 28609 heartbeater.cc:511] Master 127.25.31.190:38419 requested a full tablet report, sending...
I20260806 15:59:15.439118 28262 ts_manager.cc:194] Registered new tserver with Master: 6d07ad7432bd465ba2c51676f57f03bb (127.25.31.130:37597)
I20260806 15:59:15.439752 28262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:41051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 15:59:15.454790 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:38419
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:42607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:15.563251 28614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:15.563526 28614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:15.563601 28614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:15.567685 28614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:15.567763 28614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:15.567857 28614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:15.573063 28614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:42607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:38419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:15.574079 28614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:15.575134 28614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:15.584772 28622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:15.584846 28619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:15.584859 28620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:15.585106 28614 server_base.cc:1034] running on GCE node
I20260806 15:59:15.585583 28614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:15.586222 28614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:15.587386 28614 hybrid_clock.cc:648] HybridClock initialized: now 1786031955587374 us; error 26 us; skew 500 ppm
Aug 06 15:59:15 dist-test-slave-tm1g krb5kdc[28216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031955, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:15.590090 28614 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 15:59:15.591156 28614 webserver.cc:460] Webserver started at http://127.25.31.131:46693/ using document root <none> and password file <none>
I20260806 15:59:15.591706 28614 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:15.591797 28614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:15.592067 28614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:15.594053 28614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "925700f127e14cd68c8d004a416e250c"
format_stamp: "Formatted at 2026-08-06 15:59:15 on dist-test-slave-tm1g"
server_key: "363d0b3166a4ca2ff408a022b54ef761"
server_key_iv: "3b4492b6337220c11315879c66deac5a"
server_key_version: "encryptionkey@0"
I20260806 15:59:15.594645 28614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "925700f127e14cd68c8d004a416e250c"
format_stamp: "Formatted at 2026-08-06 15:59:15 on dist-test-slave-tm1g"
server_key: "363d0b3166a4ca2ff408a022b54ef761"
server_key_iv: "3b4492b6337220c11315879c66deac5a"
server_key_version: "encryptionkey@0"
I20260806 15:59:15.598419 28614 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 15:59:15.600922 28629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:15.602371 28614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 15:59:15.602522 28614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "925700f127e14cd68c8d004a416e250c"
format_stamp: "Formatted at 2026-08-06 15:59:15 on dist-test-slave-tm1g"
server_key: "363d0b3166a4ca2ff408a022b54ef761"
server_key_iv: "3b4492b6337220c11315879c66deac5a"
server_key_version: "encryptionkey@0"
I20260806 15:59:15.602630 28614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:15.615340 28614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:15.618005 28614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:15.618206 28614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:15.618789 28614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:15.619570 28614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:15.619644 28614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:15.619710 28614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:15.619761 28614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:15.628616 28614 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:41641
I20260806 15:59:15.628713 28748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:41641 every 8 connection(s)
I20260806 15:59:15.629467 28614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 15:59:15.630584 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 28614
I20260806 15:59:15.630669 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 15:59:15.630949 25726 external_mini_cluster.cc:1428] Setting key 1c17211b4c8ee005de228a089f64dd4b
Aug 06 15:59:15 dist-test-slave-tm1g krb5kdc[28216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031955, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:15.643231 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:15.631241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:60951 (local address 127.25.31.190:38419)
0806 15:59:15.631387 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:15.631394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:15.632331 (+ 937us) server_negotiation.cc:407] Connection header received
0806 15:59:15.633612 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:15.633614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:15.633659 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:15.633724 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:15.634800 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:15.635542 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:15.637107 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:15.637398 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:15.637521 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:15.640073 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:15.640076 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:15.640078 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:15.640100 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:15.641394 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:15.641862 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:15.641864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:15.641865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:15.641943 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:15.642226 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:15.642228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:15.642229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:15.642467 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:15.642546 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:15.642971 (+ 425us) server_negotiation.cc:299] Negotiation successful
0806 15:59:15.643061 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 15:59:15.643993 28752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:15.631662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:38419 (local address 127.25.31.131:60951)
0806 15:59:15.632177 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:15.632230 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:15.633426 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:15.633851 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:15.633857 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:15.634187 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:15.634663 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:15.634675 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:15.635699 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:15.635705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:15.636971 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:15.636983 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:15.637558 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:15.637562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:15.637784 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:15.638512 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:15.638527 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:15.639933 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:15.641510 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:15.641515 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:15.641525 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:15.641767 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:15.642021 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:15.642023 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:15.642024 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:15.642149 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:15.642558 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:15.642563 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:15.642765 (+ 202us) client_negotiation.cc:769] Sending connection context
0806 15:59:15.642923 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 15:59:15.643145 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":168,"threads_started":1}
I20260806 15:59:15.644973 28749 heartbeater.cc:347] Connected to a master server at 127.25.31.190:38419
I20260806 15:59:15.645171 28749 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:15.645581 28749 heartbeater.cc:511] Master 127.25.31.190:38419 requested a full tablet report, sending...
I20260806 15:59:15.646512 28262 ts_manager.cc:194] Registered new tserver with Master: 925700f127e14cd68c8d004a416e250c (127.25.31.131:41641)
I20260806 15:59:15.647049 28262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:60951
I20260806 15:59:15.656263 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 15:59:15.668646 28306 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:15.667328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36646 (local address 127.25.31.190:38419)
0806 15:59:15.667453 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:15.667456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:15.667676 (+ 220us) server_negotiation.cc:407] Connection header received
0806 15:59:15.667774 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:15.667776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:15.667813 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:15.667881 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:15.668501 (+ 620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36646: BlockingRecv error: recv got EOF from 127.0.0.1:36646 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 15:59:15.670702 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 28330
I20260806 15:59:15.677438 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 28473
I20260806 15:59:15.683917 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 28614
I20260806 15:59:15.689659 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 28230
2026-08-06T15:59:15Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2115 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:59:15 dist-test-slave-tm1g krb5kdc[28762](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:59:15 dist-test-slave-tm1g krb5kdc[28762](info): set up 2 sockets
Aug 06 15:59:15 dist-test-slave-tm1g krb5kdc[28762](info): commencing operation
krb5kdc: starting...
W20260806 15:59:16.734788 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 15:59:16 dist-test-slave-tm1g krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:59:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 15:59:16.852818 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:34303
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:36519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:34303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:16.960911 28776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:16.961145 28776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:16.961184 28776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:16.965040 28776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:16.965090 28776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:16.965108 28776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:16.965126 28776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:16.965139 28776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:16.970080 28776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:36519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:34303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:34303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:16.971031 28776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:16.971920 28776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:16.980028 28784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:16.980048 28781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:16.980048 28782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:16.980111 28776 server_base.cc:1034] running on GCE node
I20260806 15:59:16.980835 28776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:16.981834 28776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:16.982993 28776 hybrid_clock.cc:648] HybridClock initialized: now 1786031956982971 us; error 42 us; skew 500 ppm
Aug 06 15:59:16 dist-test-slave-tm1g krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031956, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:16.985687 28776 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 15:59:16.986619 28776 webserver.cc:460] Webserver started at http://127.25.31.190:38423/ using document root <none> and password file <none>
I20260806 15:59:16.987151 28776 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:16.987241 28776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:16.987474 28776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:16.989451 28776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "dc252a329c9e4aa886d6e81f85e27944"
format_stamp: "Formatted at 2026-08-06 15:59:16 on dist-test-slave-tm1g"
server_key: "fb20f989a9b980501f9957a852087df0"
server_key_iv: "f4b2c27d9286dc91d724a7b14599c168"
server_key_version: "encryptionkey@0"
I20260806 15:59:16.990010 28776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "dc252a329c9e4aa886d6e81f85e27944"
format_stamp: "Formatted at 2026-08-06 15:59:16 on dist-test-slave-tm1g"
server_key: "fb20f989a9b980501f9957a852087df0"
server_key_iv: "f4b2c27d9286dc91d724a7b14599c168"
server_key_version: "encryptionkey@0"
I20260806 15:59:16.993574 28776 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 15:59:16.995991 28792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:16.997225 28776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 15:59:16.997360 28776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "dc252a329c9e4aa886d6e81f85e27944"
format_stamp: "Formatted at 2026-08-06 15:59:16 on dist-test-slave-tm1g"
server_key: "fb20f989a9b980501f9957a852087df0"
server_key_iv: "f4b2c27d9286dc91d724a7b14599c168"
server_key_version: "encryptionkey@0"
I20260806 15:59:16.997475 28776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:17.018967 28776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:17.021901 28776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:17.022114 28776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:17.029170 28776 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:34303
I20260806 15:59:17.029459 28851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:34303 every 8 connection(s)
I20260806 15:59:17.030143 28776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:17.033603 28852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:17.038388 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 28776
I20260806 15:59:17.038518 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:17.038812 25726 external_mini_cluster.cc:1428] Setting key d10ad3a38393aa7a35b37d82782257da
I20260806 15:59:17.041038 28852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944: Bootstrap starting.
I20260806 15:59:17.044123 28852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:17.045120 28852 log.cc:824] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:17.047559 28852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944: No bootstrap required, opened a new log
Aug 06 15:59:17 dist-test-slave-tm1g krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:17.052575 28852 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc252a329c9e4aa886d6e81f85e27944" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34303 } }
I20260806 15:59:17.052881 28852 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:17.052951 28852 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc252a329c9e4aa886d6e81f85e27944, State: Initialized, Role: FOLLOWER
I20260806 15:59:17.053510 28852 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc252a329c9e4aa886d6e81f85e27944" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34303 } }
I20260806 15:59:17.053701 28852 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:17.053794 28852 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:17.053910 28852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:17.055235 28852 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc252a329c9e4aa886d6e81f85e27944" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34303 } }
I20260806 15:59:17.055716 28852 leader_election.cc:302] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc252a329c9e4aa886d6e81f85e27944; no voters:
I20260806 15:59:17.056049 28855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:17.040596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46040 (local address 127.25.31.190:34303)
0806 15:59:17.041065 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:17.041087 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:17.041134 (+ 47us) server_negotiation.cc:407] Connection header received
0806 15:59:17.042012 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:17.042030 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:17.042444 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:17.042763 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:17.043592 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:17.045150 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:17.046554 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:17.047019 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:17.047189 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:17.049999 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:17.050005 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:17.050021 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:17.050058 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:17.052080 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:17.052532 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:17.052537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:17.052543 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:17.052659 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:17.053000 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:17.053003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:17.053005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:17.053521 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:17.053744 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:17.054585 (+ 841us) server_negotiation.cc:299] Negotiation successful
0806 15:59:17.054906 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":133,"threads_started":1}
I20260806 15:59:17.056145 28852 leader_election.cc:288] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:17.056686 28858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:17.056849 28858 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [term 1 LEADER]: Becoming Leader. State: Replica: dc252a329c9e4aa886d6e81f85e27944, State: Running, Role: LEADER
I20260806 15:59:17.057169 28858 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc252a329c9e4aa886d6e81f85e27944" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34303 } }
I20260806 15:59:17.057597 28852 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:17.058271 28858 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc252a329c9e4aa886d6e81f85e27944. Latest consensus state: current_term: 1 leader_uuid: "dc252a329c9e4aa886d6e81f85e27944" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc252a329c9e4aa886d6e81f85e27944" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34303 } } }
I20260806 15:59:17.058401 28858 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:17.058771 28864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:17.059391 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:59:17.060410 28857 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc252a329c9e4aa886d6e81f85e27944" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc252a329c9e4aa886d6e81f85e27944" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34303 } } }
I20260806 15:59:17.060514 28857 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:17.062186 28864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:17.068290 28864 catalog_manager.cc:1347] Generated new cluster ID: 5537a371c62845629697e97e7a34bc02
I20260806 15:59:17.068413 28864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:17.080495 28855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:17.074745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46052 (local address 127.25.31.190:34303)
0806 15:59:17.074859 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:17.074863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:17.075318 (+ 455us) server_negotiation.cc:407] Connection header received
0806 15:59:17.075449 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:17.075452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:17.075500 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:17.075593 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:17.076109 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:17.076921 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:17.078036 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:17.078419 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:17.078487 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:17.079511 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:17.079516 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:17.079518 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:17.080026 (+ 508us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:17.080058 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:17.080076 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:17.080130 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:17.080288 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 15:59:17.080345 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 15:59:17.087464 28864 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:17.088357 28864 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:17.093920 28864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc252a329c9e4aa886d6e81f85e27944: Generated new TSK 0
I20260806 15:59:17.094660 28864 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:59:17.100234 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 28776
2026-08-06T15:59:17Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1410 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:59:17 dist-test-slave-tm1g krb5kdc[28888](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:59:17 dist-test-slave-tm1g krb5kdc[28888](info): set up 2 sockets
Aug 06 15:59:17 dist-test-slave-tm1g krb5kdc[28888](info): commencing operation
krb5kdc: starting...
W20260806 15:59:18.154537 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 15:59:18 dist-test-slave-tm1g krb5kdc[28888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:59:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 15:59:18.272930 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:40465
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:40465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:18.380800 28901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:18.381083 28901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:18.381168 28901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:18.385319 28901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:18.385398 28901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:18.385435 28901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:18.385490 28901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:18.385527 28901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:18.390714 28901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:40465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:40465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:18.391691 28901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:18.392669 28901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:18.400746 28910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:18.400758 28907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:18.400758 28908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:18.401206 28901 server_base.cc:1034] running on GCE node
I20260806 15:59:18.401748 28901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:18.402853 28901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:18.404053 28901 hybrid_clock.cc:648] HybridClock initialized: now 1786031958404034 us; error 29 us; skew 500 ppm
Aug 06 15:59:18 dist-test-slave-tm1g krb5kdc[28888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031958, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:18.406759 28901 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 15:59:18.407606 28901 webserver.cc:460] Webserver started at http://127.25.31.190:39633/ using document root <none> and password file <none>
I20260806 15:59:18.408150 28901 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:18.408242 28901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:18.408478 28901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:18.410491 28901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "02ba117f8d264896aabcf462fca0229e"
format_stamp: "Formatted at 2026-08-06 15:59:18 on dist-test-slave-tm1g"
server_key: "4b9e6a7d6e07885b86db6a08cc1d496c"
server_key_iv: "9cb6b1c0904b7f0f9a83754ea81ec515"
server_key_version: "encryptionkey@0"
I20260806 15:59:18.411051 28901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "02ba117f8d264896aabcf462fca0229e"
format_stamp: "Formatted at 2026-08-06 15:59:18 on dist-test-slave-tm1g"
server_key: "4b9e6a7d6e07885b86db6a08cc1d496c"
server_key_iv: "9cb6b1c0904b7f0f9a83754ea81ec515"
server_key_version: "encryptionkey@0"
I20260806 15:59:18.414589 28901 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 15:59:18.416884 28917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:18.418023 28901 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:18.418174 28901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "02ba117f8d264896aabcf462fca0229e"
format_stamp: "Formatted at 2026-08-06 15:59:18 on dist-test-slave-tm1g"
server_key: "4b9e6a7d6e07885b86db6a08cc1d496c"
server_key_iv: "9cb6b1c0904b7f0f9a83754ea81ec515"
server_key_version: "encryptionkey@0"
I20260806 15:59:18.418288 28901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:18.437022 28901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:18.439682 28901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:18.439877 28901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:18.446457 28901 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:40465
I20260806 15:59:18.446755 28971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:40465 every 8 connection(s)
I20260806 15:59:18.447374 28901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:18.448405 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 28901
I20260806 15:59:18.448534 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:18.448837 25726 external_mini_cluster.cc:1428] Setting key 61b44057442da271acf14022e6376346
I20260806 15:59:18.450660 28972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:18.456948 28972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e: Bootstrap starting.
Aug 06 15:59:18 dist-test-slave-tm1g krb5kdc[28888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:18.459208 28972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:18.459895 28972 log.cc:824] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:18.461679 28972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e: No bootstrap required, opened a new log
I20260806 15:59:18.463333 28975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:18.450171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38974 (local address 127.25.31.190:40465)
0806 15:59:18.450612 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:18.450632 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:18.450666 (+ 34us) server_negotiation.cc:407] Connection header received
0806 15:59:18.451349 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:18.451365 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:18.451663 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:18.451941 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:18.452841 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:18.453829 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:18.455046 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:18.455509 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:18.455633 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:18.458556 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:18.458562 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:18.458574 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:18.458600 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:18.460427 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:18.460897 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:18.460901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:18.460906 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:18.461012 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:18.461296 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:18.461299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:18.461301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:18.461669 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:18.461897 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:18.462191 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 15:59:18.462491 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":179,"threads_started":1}
I20260806 15:59:18.464633 28972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ba117f8d264896aabcf462fca0229e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40465 } }
I20260806 15:59:18.464861 28972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:18.464959 28972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02ba117f8d264896aabcf462fca0229e, State: Initialized, Role: FOLLOWER
I20260806 15:59:18.465440 28972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ba117f8d264896aabcf462fca0229e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40465 } }
I20260806 15:59:18.465623 28972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:18.465691 28972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:18.465785 28972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:18.466744 28972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ba117f8d264896aabcf462fca0229e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40465 } }
I20260806 15:59:18.467064 28972 leader_election.cc:302] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02ba117f8d264896aabcf462fca0229e; no voters:
I20260806 15:59:18.467345 28972 leader_election.cc:288] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:18.467480 28977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:18.468037 28977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [term 1 LEADER]: Becoming Leader. State: Replica: 02ba117f8d264896aabcf462fca0229e, State: Running, Role: LEADER
I20260806 15:59:18.468350 28977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ba117f8d264896aabcf462fca0229e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40465 } }
I20260806 15:59:18.468559 28972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:18.469385 28979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02ba117f8d264896aabcf462fca0229e. Latest consensus state: current_term: 1 leader_uuid: "02ba117f8d264896aabcf462fca0229e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ba117f8d264896aabcf462fca0229e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40465 } } }
I20260806 15:59:18.469524 28979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:18.469805 28978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02ba117f8d264896aabcf462fca0229e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ba117f8d264896aabcf462fca0229e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40465 } } }
I20260806 15:59:18.469895 28978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:18.470005 28987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:18.472951 28987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:18.478472 28987 catalog_manager.cc:1347] Generated new cluster ID: d3ed65519e2f4ae387a60260a06733b7
I20260806 15:59:18.478550 28987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:18.490422 28987 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:18.491271 28987 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:18.500114 28987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02ba117f8d264896aabcf462fca0229e: Generated new TSK 0
I20260806 15:59:18.500737 28987 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 15:59:18.533475 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:40465
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:18.642166 29001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:18.642418 29001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:18.642483 29001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:18.647037 29001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:18.647126 29001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:18.647234 29001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:18.653177 29001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:40465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:18.654147 29001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:18.655162 29001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:18.664538 29009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:18.664552 29007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:18.664575 29006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:18.664588 29001 server_base.cc:1034] running on GCE node
I20260806 15:59:18.665269 29001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:18.665887 29001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:18.667106 29001 hybrid_clock.cc:648] HybridClock initialized: now 1786031958667092 us; error 23 us; skew 500 ppm
Aug 06 15:59:18 dist-test-slave-tm1g krb5kdc[28888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031958, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:18.674818 29001 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 15:59:18.675709 29001 webserver.cc:460] Webserver started at http://127.25.31.129:36839/ using document root <none> and password file <none>
I20260806 15:59:18.676263 29001 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:18.676354 29001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:18.676584 29001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:18.678699 29001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "b5e37fbef846453eba51fcf9b504cfa4"
format_stamp: "Formatted at 2026-08-06 15:59:18 on dist-test-slave-tm1g"
server_key: "544651209cbc8527cc9359729e4b8c35"
server_key_iv: "128aee6496ba9196f09591031b100922"
server_key_version: "encryptionkey@0"
I20260806 15:59:18.679276 29001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "b5e37fbef846453eba51fcf9b504cfa4"
format_stamp: "Formatted at 2026-08-06 15:59:18 on dist-test-slave-tm1g"
server_key: "544651209cbc8527cc9359729e4b8c35"
server_key_iv: "128aee6496ba9196f09591031b100922"
server_key_version: "encryptionkey@0"
I20260806 15:59:18.682904 29001 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 15:59:18.685238 29016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:18.686478 29001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:18.686604 29001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "b5e37fbef846453eba51fcf9b504cfa4"
format_stamp: "Formatted at 2026-08-06 15:59:18 on dist-test-slave-tm1g"
server_key: "544651209cbc8527cc9359729e4b8c35"
server_key_iv: "128aee6496ba9196f09591031b100922"
server_key_version: "encryptionkey@0"
I20260806 15:59:18.686709 29001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:18.719148 29001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:18.722602 29001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:18.722806 29001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:18.723392 29001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:18.724196 29001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:18.724267 29001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:18.724334 29001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:18.724385 29001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:18.733260 29001 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:45837
I20260806 15:59:18.733304 29133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:45837 every 8 connection(s)
I20260806 15:59:18.734161 29001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:18.740077 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 29001
I20260806 15:59:18.740203 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 15:59:18.740485 25726 external_mini_cluster.cc:1428] Setting key 7e6c7b0ab696af0de6b97358b461a61f
Aug 06 15:59:18 dist-test-slave-tm1g krb5kdc[28888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031958, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:18.747957 28975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:18.736014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:35673 (local address 127.25.31.190:40465)
0806 15:59:18.736168 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:18.736173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:18.736978 (+ 805us) server_negotiation.cc:407] Connection header received
0806 15:59:18.738266 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:18.738269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:18.738342 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:18.738433 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:18.739511 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:18.740159 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:18.741374 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:18.741620 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:18.741733 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:18.744360 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:18.744368 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:18.744371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:18.744393 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:18.745875 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:18.746435 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:18.746439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:18.746441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:18.746524 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:18.746862 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:18.746867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:18.746869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:18.747052 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:18.747160 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:18.747643 (+ 483us) server_negotiation.cc:299] Negotiation successful
0806 15:59:18.747772 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 15:59:18.748704 29136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:18.736393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40465 (local address 127.25.31.129:35673)
0806 15:59:18.736828 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:18.736873 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:18.738060 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:18.738568 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:18.738574 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:18.738906 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:18.739373 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:18.739386 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:18.740329 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:18.740332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:18.741247 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:18.741257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:18.741772 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:18.741776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:18.741940 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:18.742545 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:18.742561 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:18.744182 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:18.746014 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:18.746019 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:18.746032 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:18.746292 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:18.746617 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:18.746621 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:18.746623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:18.746764 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:18.747160 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:18.747167 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:18.747414 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 15:59:18.747600 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 15:59:18.747906 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":107,"threads_started":1}
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
I20260806 15:59:18.750017 29134 heartbeater.cc:347] Connected to a master server at 127.25.31.190:40465
I20260806 15:59:18.750255 29134 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:18.750780 29134 heartbeater.cc:511] Master 127.25.31.190:40465 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
I20260806 15:59:18.752313 28935 ts_manager.cc:194] Registered new tserver with Master: b5e37fbef846453eba51fcf9b504cfa4 (127.25.31.129:45837)
I20260806 15:59:18.753916 28935 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:35673
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 15:59:18.767249 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:40465
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:18.874199 29140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:18.874493 29140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:18.874568 29140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:18.878531 29140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:18.878602 29140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:18.878690 29140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:18.883703 29140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:40465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:18.884670 29140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:18.885653 29140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:18.895160 29146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:18.895160 29148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:18.895215 29145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:18.895365 29140 server_base.cc:1034] running on GCE node
I20260806 15:59:18.895884 29140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:18.896577 29140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:18.897735 29140 hybrid_clock.cc:648] HybridClock initialized: now 1786031958897724 us; error 24 us; skew 500 ppm
Aug 06 15:59:18 dist-test-slave-tm1g krb5kdc[28888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031958, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:18.900326 29140 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 15:59:18.901113 29140 webserver.cc:460] Webserver started at http://127.25.31.130:34509/ using document root <none> and password file <none>
I20260806 15:59:18.901649 29140 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:18.901736 29140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:18.901930 29140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:18.903944 29140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "1c2575b3b32849f4b644247915885c62"
format_stamp: "Formatted at 2026-08-06 15:59:18 on dist-test-slave-tm1g"
server_key: "0353319bcb5c330c073c63024a940399"
server_key_iv: "18fc9ab1ed555f1703c85770cb20ac1b"
server_key_version: "encryptionkey@0"
I20260806 15:59:18.904500 29140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "1c2575b3b32849f4b644247915885c62"
format_stamp: "Formatted at 2026-08-06 15:59:18 on dist-test-slave-tm1g"
server_key: "0353319bcb5c330c073c63024a940399"
server_key_iv: "18fc9ab1ed555f1703c85770cb20ac1b"
server_key_version: "encryptionkey@0"
I20260806 15:59:18.908177 29140 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 15:59:18.910521 29155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:18.911650 29140 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:18.911782 29140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "1c2575b3b32849f4b644247915885c62"
format_stamp: "Formatted at 2026-08-06 15:59:18 on dist-test-slave-tm1g"
server_key: "0353319bcb5c330c073c63024a940399"
server_key_iv: "18fc9ab1ed555f1703c85770cb20ac1b"
server_key_version: "encryptionkey@0"
I20260806 15:59:18.911887 29140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:18.923894 29140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:18.926578 29140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:18.926765 29140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:18.927299 29140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:18.928069 29140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:18.928139 29140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:18.928207 29140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:18.928261 29140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:18.937012 29140 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:44547
I20260806 15:59:18.937044 29269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:44547 every 8 connection(s)
I20260806 15:59:18.937822 29140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:18.943069 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 29140
I20260806 15:59:18.943212 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 15:59:18.943502 25726 external_mini_cluster.cc:1428] Setting key 29791bb1e17619262d16492860be29b3
Aug 06 15:59:18 dist-test-slave-tm1g krb5kdc[28888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031958, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:18.951817 28975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:18.939439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:51931 (local address 127.25.31.190:40465)
0806 15:59:18.939593 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:18.939597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:18.940369 (+ 772us) server_negotiation.cc:407] Connection header received
0806 15:59:18.941780 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:18.941784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:18.941843 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:18.941949 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:18.943126 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:18.943812 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:18.945035 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:18.945253 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:18.945341 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:18.948000 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:18.948006 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:18.948009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:18.948038 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:18.949745 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:18.950221 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:18.950227 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:18.950232 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:18.950347 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:18.950667 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:18.950673 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:18.950677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:18.950939 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:18.951048 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:18.951483 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 15:59:18.951630 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 15:59:18.952450 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:18.939782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40465 (local address 127.25.31.130:51931)
0806 15:59:18.940222 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:18.940263 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:18.941533 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:18.942120 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:18.942127 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:18.942519 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:18.942980 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:18.942993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:18.943973 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:18.943976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:18.944903 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:18.944914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:18.945415 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:18.945421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:18.945650 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:18.946224 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:18.946250 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:18.947834 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:18.949868 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:18.949872 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:18.949885 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:18.950117 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:18.950443 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:18.950445 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:18.950446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:18.950570 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:18.951049 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:18.951053 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:18.951254 (+ 201us) client_negotiation.cc:769] Sending connection context
0806 15:59:18.951410 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 15:59:18.951635 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
I20260806 15:59:18.953540 29270 heartbeater.cc:347] Connected to a master server at 127.25.31.190:40465
I20260806 15:59:18.953768 29270 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:18.954221 29270 heartbeater.cc:511] Master 127.25.31.190:40465 requested a full tablet report, sending...
I20260806 15:59:18.955243 28935 ts_manager.cc:194] Registered new tserver with Master: 1c2575b3b32849f4b644247915885c62 (127.25.31.130:44547)
I20260806 15:59:18.955873 28935 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:51931
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 15:59:18.970652 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:40465
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:19.078114 29277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:19.078400 29277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:19.078490 29277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:19.082441 29277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:19.082521 29277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:19.082618 29277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:19.087685 29277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:40465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:19.088646 29277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:19.089596 29277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:19.099031 29282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:19.099114 29277 server_base.cc:1034] running on GCE node
W20260806 15:59:19.099036 29283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:19.099031 29285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:19.099702 29277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:19.100283 29277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:19.101435 29277 hybrid_clock.cc:648] HybridClock initialized: now 1786031959101417 us; error 31 us; skew 500 ppm
Aug 06 15:59:19 dist-test-slave-tm1g krb5kdc[28888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031959, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:19.104460 29277 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 15:59:19.105326 29277 webserver.cc:460] Webserver started at http://127.25.31.131:41623/ using document root <none> and password file <none>
I20260806 15:59:19.105880 29277 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:19.105969 29277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:19.106176 29277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:19.108191 29277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "2fb1f3962787436ba31b4b074cb6d607"
format_stamp: "Formatted at 2026-08-06 15:59:19 on dist-test-slave-tm1g"
server_key: "d27ef1ae15bc09ff4ee8c832c1ab3671"
server_key_iv: "328e795fa9091933d086e5fd126d5c99"
server_key_version: "encryptionkey@0"
I20260806 15:59:19.108752 29277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "2fb1f3962787436ba31b4b074cb6d607"
format_stamp: "Formatted at 2026-08-06 15:59:19 on dist-test-slave-tm1g"
server_key: "d27ef1ae15bc09ff4ee8c832c1ab3671"
server_key_iv: "328e795fa9091933d086e5fd126d5c99"
server_key_version: "encryptionkey@0"
I20260806 15:59:19.112311 29277 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 15:59:19.114681 29292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:19.115865 29277 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 15:59:19.115989 29277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "2fb1f3962787436ba31b4b074cb6d607"
format_stamp: "Formatted at 2026-08-06 15:59:19 on dist-test-slave-tm1g"
server_key: "d27ef1ae15bc09ff4ee8c832c1ab3671"
server_key_iv: "328e795fa9091933d086e5fd126d5c99"
server_key_version: "encryptionkey@0"
I20260806 15:59:19.116097 29277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:19.142114 29277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:19.144932 29277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:19.145138 29277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:19.145737 29277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:19.146576 29277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:19.146651 29277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:19.146726 29277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:19.146770 29277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:19.155830 29277 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:33709
I20260806 15:59:19.155865 29410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:33709 every 8 connection(s)
I20260806 15:59:19.156715 29277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
Aug 06 15:59:19 dist-test-slave-tm1g krb5kdc[28888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031959, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:19.166342 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 29277
I20260806 15:59:19.166482 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 15:59:19.166783 25726 external_mini_cluster.cc:1428] Setting key f854db843f9623d564c2e218eb811c5b
I20260806 15:59:19.170750 28975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:19.158525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:35485 (local address 127.25.31.190:40465)
0806 15:59:19.158682 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:19.158687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:19.159579 (+ 892us) server_negotiation.cc:407] Connection header received
0806 15:59:19.160904 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:19.160907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:19.160955 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:19.161039 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:19.162157 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:19.162751 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:19.163878 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:19.164097 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:19.164174 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:19.166672 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:19.166675 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:19.166677 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:19.166701 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:19.168478 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:19.169159 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:19.169164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:19.169165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:19.169253 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:19.169607 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:19.169612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:19.169614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:19.169796 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:19.169903 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:19.170459 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 15:59:19.170582 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 15:59:19.171496 29414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:19.158956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40465 (local address 127.25.31.131:35485)
0806 15:59:19.159424 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:19.159468 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:19.160703 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:19.161193 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:19.161200 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:19.161555 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:19.162015 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:19.162027 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:19.162875 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:19.162878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:19.163765 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:19.163775 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:19.164233 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:19.164236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:19.164391 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:19.164916 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:19.164931 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:19.166496 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:19.168582 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:19.168588 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:19.168601 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:19.169005 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:19.169348 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:19.169352 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:19.169354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:19.169508 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:19.169902 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:19.169908 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:19.170205 (+ 297us) client_negotiation.cc:769] Sending connection context
0806 15:59:19.170426 (+ 221us) client_negotiation.cc:240] Negotiation successful
0806 15:59:19.170727 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":120,"threads_started":1}
I20260806 15:59:19.172516 29411 heartbeater.cc:347] Connected to a master server at 127.25.31.190:40465
I20260806 15:59:19.172720 29411 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:19.173121 29411 heartbeater.cc:511] Master 127.25.31.190:40465 requested a full tablet report, sending...
I20260806 15:59:19.174015 28935 ts_manager.cc:194] Registered new tserver with Master: 2fb1f3962787436ba31b4b074cb6d607 (127.25.31.131:33709)
I20260806 15:59:19.174592 28935 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:35485
I20260806 15:59:19.180604 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 15:59:19 dist-test-slave-tm1g krb5kdc[28888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031959, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 15:59:19 dist-test-slave-tm1g krb5kdc[28888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031959, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:19.206122 28975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:19.197656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38976 (local address 127.25.31.190:40465)
0806 15:59:19.197778 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:19.197782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:19.197990 (+ 208us) server_negotiation.cc:407] Connection header received
0806 15:59:19.198165 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:19.198168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:19.198208 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:19.198278 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:19.198993 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:19.199516 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:19.200307 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:19.200597 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:19.200732 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:19.203169 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:19.203173 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:19.203175 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:19.203200 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:19.204421 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:19.204846 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:19.204850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:19.204851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:19.204935 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:19.205209 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:19.205214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:19.205215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:19.205447 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:19.205591 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:19.205825 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 15:59:19.205945 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 15:59:19.206593 28935 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38976
I20260806 15:59:19.207424 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 29001
I20260806 15:59:19.216259 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 29140
I20260806 15:59:19.222227 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 29277
I20260806 15:59:19.227834 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 28901
2026-08-06T15:59:19Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2127 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:59:19 dist-test-slave-tm1g krb5kdc[29424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:59:19 dist-test-slave-tm1g krb5kdc[29424](info): set up 2 sockets
Aug 06 15:59:19 dist-test-slave-tm1g krb5kdc[29424](info): commencing operation
krb5kdc: starting...
W20260806 15:59:20.270747 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 15:59:20 dist-test-slave-tm1g krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:59:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 15:59:20.393497 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:39565
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:39565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:20.501775 29437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:20.502019 29437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:20.502059 29437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:20.505887 29437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:20.505934 29437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:20.505952 29437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:20.505968 29437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:20.505983 29437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:20.510944 29437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:39565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:39565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:20.511829 29437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:20.512728 29437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:20.520865 29443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:20.520841 29442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:20.520952 29445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:20.521099 29437 server_base.cc:1034] running on GCE node
I20260806 15:59:20.521796 29437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:20.522989 29437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:20.524183 29437 hybrid_clock.cc:648] HybridClock initialized: now 1786031960524154 us; error 38 us; skew 500 ppm
Aug 06 15:59:20 dist-test-slave-tm1g krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031960, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:20.527021 29437 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 15:59:20.527911 29437 webserver.cc:460] Webserver started at http://127.25.31.190:37027/ using document root <none> and password file <none>
I20260806 15:59:20.528426 29437 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:20.528489 29437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:20.528725 29437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:20.530855 29437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "7aafe39b4f934fe39afca3f91d912628"
format_stamp: "Formatted at 2026-08-06 15:59:20 on dist-test-slave-tm1g"
server_key: "9427a93e241792fae5b3889cc2d8b31d"
server_key_iv: "6cd5154593aa7dc18af72aa15c3b25ed"
server_key_version: "encryptionkey@0"
I20260806 15:59:20.531387 29437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "7aafe39b4f934fe39afca3f91d912628"
format_stamp: "Formatted at 2026-08-06 15:59:20 on dist-test-slave-tm1g"
server_key: "9427a93e241792fae5b3889cc2d8b31d"
server_key_iv: "6cd5154593aa7dc18af72aa15c3b25ed"
server_key_version: "encryptionkey@0"
I20260806 15:59:20.535048 29437 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 15:59:20.537279 29452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:20.538524 29437 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:20.538642 29437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "7aafe39b4f934fe39afca3f91d912628"
format_stamp: "Formatted at 2026-08-06 15:59:20 on dist-test-slave-tm1g"
server_key: "9427a93e241792fae5b3889cc2d8b31d"
server_key_iv: "6cd5154593aa7dc18af72aa15c3b25ed"
server_key_version: "encryptionkey@0"
I20260806 15:59:20.538775 29437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:20.552551 29437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:20.555228 29437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:20.555370 29437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:20.562338 29437 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:39565
I20260806 15:59:20.562353 29509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:39565 every 8 connection(s)
I20260806 15:59:20.563248 29437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:20.565798 29510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:20.569345 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 29437
I20260806 15:59:20.569438 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:20.569737 25726 external_mini_cluster.cc:1428] Setting key be0d83140e3db8d0cf99a2b6e8f29937
I20260806 15:59:20.571352 29510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628: Bootstrap starting.
I20260806 15:59:20.573406 29510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:20.574122 29510 log.cc:824] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:20.575929 29510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628: No bootstrap required, opened a new log
Aug 06 15:59:20 dist-test-slave-tm1g krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:20.579681 29510 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aafe39b4f934fe39afca3f91d912628" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 39565 } }
I20260806 15:59:20.579972 29510 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:20.580037 29510 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aafe39b4f934fe39afca3f91d912628, State: Initialized, Role: FOLLOWER
I20260806 15:59:20.580508 29510 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aafe39b4f934fe39afca3f91d912628" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 39565 } }
I20260806 15:59:20.580628 29510 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:20.580711 29510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:20.580830 29510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:20.581660 29510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aafe39b4f934fe39afca3f91d912628" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 39565 } }
I20260806 15:59:20.581995 29510 leader_election.cc:302] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7aafe39b4f934fe39afca3f91d912628; no voters:
I20260806 15:59:20.582345 29510 leader_election.cc:288] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:20.582438 29515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:20.582631 29515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [term 1 LEADER]: Becoming Leader. State: Replica: 7aafe39b4f934fe39afca3f91d912628, State: Running, Role: LEADER
I20260806 15:59:20.582931 29515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aafe39b4f934fe39afca3f91d912628" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 39565 } }
I20260806 15:59:20.583606 29510 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:20.583950 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:20.571111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59854 (local address 127.25.31.190:39565)
0806 15:59:20.571560 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:20.571574 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:20.571608 (+ 34us) server_negotiation.cc:407] Connection header received
0806 15:59:20.572215 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:20.572230 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:20.572525 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:20.572841 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:20.573671 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:20.574885 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:20.575764 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:20.576300 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:20.576477 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:20.578945 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:20.578950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:20.578962 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:20.578988 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:20.580909 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:20.581332 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:20.581338 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:20.581343 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:20.581461 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:20.581747 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:20.581752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:20.581754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:20.582195 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:20.582415 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:20.582729 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 15:59:20.583045 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":129,"threads_started":1}
I20260806 15:59:20.584168 29517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7aafe39b4f934fe39afca3f91d912628. Latest consensus state: current_term: 1 leader_uuid: "7aafe39b4f934fe39afca3f91d912628" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aafe39b4f934fe39afca3f91d912628" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 39565 } } }
I20260806 15:59:20.584282 29517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:20.584677 29516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7aafe39b4f934fe39afca3f91d912628" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aafe39b4f934fe39afca3f91d912628" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 39565 } } }
I20260806 15:59:20.584785 29516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:20.585537 29520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:20.588598 29520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:20.594265 29520 catalog_manager.cc:1347] Generated new cluster ID: d0399b0b642544919cf997d13b5b6804
I20260806 15:59:20.594354 29520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:20.601897 29520 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:20.602809 29520 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:20.610707 29520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7aafe39b4f934fe39afca3f91d912628: Generated new TSK 0
I20260806 15:59:20.611379 29520 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 15:59:20.643785 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:39565
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:20.771730 29539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:20.771971 29539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:20.772014 29539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:20.775883 29539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:20.775931 29539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:20.775986 29539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:20.781003 29539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:39565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:20.781908 29539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:20.782847 29539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:20.792237 29545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:20.792289 29548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:20.792290 29546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:20.792590 29539 server_base.cc:1034] running on GCE node
I20260806 15:59:20.792971 29539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:20.793593 29539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:20.794780 29539 hybrid_clock.cc:648] HybridClock initialized: now 1786031960794770 us; error 36 us; skew 500 ppm
Aug 06 15:59:20 dist-test-slave-tm1g krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031960, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:20.797449 29539 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 15:59:20.798434 29539 webserver.cc:460] Webserver started at http://127.25.31.129:34957/ using document root <none> and password file <none>
I20260806 15:59:20.798988 29539 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:20.799077 29539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:20.799307 29539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:20.801373 29539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "17f9463010f54dd2bf745b3a5a530e4d"
format_stamp: "Formatted at 2026-08-06 15:59:20 on dist-test-slave-tm1g"
server_key: "bb10f9349b447b2c7ea8a7cfadffbf1d"
server_key_iv: "b75434ce9be06327c56ba7b0692ceba9"
server_key_version: "encryptionkey@0"
I20260806 15:59:20.801952 29539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "17f9463010f54dd2bf745b3a5a530e4d"
format_stamp: "Formatted at 2026-08-06 15:59:20 on dist-test-slave-tm1g"
server_key: "bb10f9349b447b2c7ea8a7cfadffbf1d"
server_key_iv: "b75434ce9be06327c56ba7b0692ceba9"
server_key_version: "encryptionkey@0"
I20260806 15:59:20.805649 29539 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 15:59:20.808139 29556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:20.809341 29539 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:20.809476 29539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "17f9463010f54dd2bf745b3a5a530e4d"
format_stamp: "Formatted at 2026-08-06 15:59:20 on dist-test-slave-tm1g"
server_key: "bb10f9349b447b2c7ea8a7cfadffbf1d"
server_key_iv: "b75434ce9be06327c56ba7b0692ceba9"
server_key_version: "encryptionkey@0"
I20260806 15:59:20.809592 29539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:20.831192 29539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:20.834246 29539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:20.834491 29539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:20.835094 29539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:20.835934 29539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:20.836005 29539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:20.836077 29539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:20.836128 29539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:20.845144 29539 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:42259
I20260806 15:59:20.845224 29677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:42259 every 8 connection(s)
I20260806 15:59:20.846060 29539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:20.851091 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 29539
I20260806 15:59:20.851218 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 15:59:20.851543 25726 external_mini_cluster.cc:1428] Setting key 913ad31eb16e510654828de587d59537
Aug 06 15:59:20 dist-test-slave-tm1g krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031960, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:20.861048 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:20.847877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:55595 (local address 127.25.31.190:39565)
0806 15:59:20.848047 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:20.848052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:20.848820 (+ 768us) server_negotiation.cc:407] Connection header received
0806 15:59:20.850182 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:20.850185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:20.850231 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:20.850343 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:20.851772 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:20.852502 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:20.854040 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:20.854265 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:20.854444 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:20.857219 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:20.857226 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:20.857232 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:20.857266 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:20.858835 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:20.859487 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:20.859490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:20.859491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:20.859566 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:20.859962 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:20.859964 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:20.859965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:20.860125 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:20.860197 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:20.860816 (+ 619us) server_negotiation.cc:299] Negotiation successful
0806 15:59:20.860922 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":70}
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
I20260806 15:59:20.862212 29680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:20.848215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:39565 (local address 127.25.31.129:55595)
0806 15:59:20.848663 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:20.848705 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:20.849977 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:20.850528 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:20.850535 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:20.850947 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:20.851618 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:20.851639 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:20.852625 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:20.852627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:20.853905 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:20.853921 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:20.854500 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:20.854507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:20.854748 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:20.855421 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:20.855437 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:20.857049 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:20.858958 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:20.858967 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:20.858984 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:20.859385 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:20.859658 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:20.859664 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:20.859667 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:20.859859 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:20.860234 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:20.860241 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:20.860571 (+ 330us) client_negotiation.cc:769] Sending connection context
0806 15:59:20.860793 (+ 222us) client_negotiation.cc:240] Negotiation successful
0806 15:59:20.861124 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":113,"threads_started":1}
I20260806 15:59:20.863452 29678 heartbeater.cc:347] Connected to a master server at 127.25.31.190:39565
I20260806 15:59:20.863660 29678 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:20.864092 29678 heartbeater.cc:511] Master 127.25.31.190:39565 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
I20260806 15:59:20.865679 29473 ts_manager.cc:194] Registered new tserver with Master: 17f9463010f54dd2bf745b3a5a530e4d (127.25.31.129:42259)
I20260806 15:59:20.866888 29473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:55595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 15:59:20.881160 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:39565
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:20.988535 29683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:20.988801 29683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:20.988878 29683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:20.992867 29683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:20.992949 29683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:20.993039 29683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:20.998023 29683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:39565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:20.999007 29683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:20.999982 29683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:21.009385 29691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:21.009399 29694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:21.009469 29683 server_base.cc:1034] running on GCE node
W20260806 15:59:21.009361 29692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:21.010068 29683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:21.010739 29683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:21.011916 29683 hybrid_clock.cc:648] HybridClock initialized: now 1786031961011881 us; error 41 us; skew 500 ppm
Aug 06 15:59:21 dist-test-slave-tm1g krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031961, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:21.014894 29683 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 15:59:21.015693 29683 webserver.cc:460] Webserver started at http://127.25.31.130:42753/ using document root <none> and password file <none>
I20260806 15:59:21.016263 29683 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:21.016327 29683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:21.016505 29683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:21.018538 29683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "283a58b05a3d49f1adca53548023b017"
format_stamp: "Formatted at 2026-08-06 15:59:21 on dist-test-slave-tm1g"
server_key: "90cc1c786045dd6097e0304b14b10fb3"
server_key_iv: "2f0691cd0ddcb296af15f6a45cdad9de"
server_key_version: "encryptionkey@0"
I20260806 15:59:21.019042 29683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "283a58b05a3d49f1adca53548023b017"
format_stamp: "Formatted at 2026-08-06 15:59:21 on dist-test-slave-tm1g"
server_key: "90cc1c786045dd6097e0304b14b10fb3"
server_key_iv: "2f0691cd0ddcb296af15f6a45cdad9de"
server_key_version: "encryptionkey@0"
I20260806 15:59:21.022616 29683 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 15:59:21.024955 29701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:21.026173 29683 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:21.026381 29683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "283a58b05a3d49f1adca53548023b017"
format_stamp: "Formatted at 2026-08-06 15:59:21 on dist-test-slave-tm1g"
server_key: "90cc1c786045dd6097e0304b14b10fb3"
server_key_iv: "2f0691cd0ddcb296af15f6a45cdad9de"
server_key_version: "encryptionkey@0"
I20260806 15:59:21.026504 29683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:21.044760 29683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:21.047806 29683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:21.047997 29683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:21.048545 29683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:21.049330 29683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:21.049397 29683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:21.049463 29683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:21.049507 29683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:21.057996 29683 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:42813
I20260806 15:59:21.058022 29821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:42813 every 8 connection(s)
I20260806 15:59:21.058888 29683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:21.067111 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 29683
I20260806 15:59:21.067245 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 15:59:21.067586 25726 external_mini_cluster.cc:1428] Setting key bae636524a6ff74abdca1a613e9b2599
Aug 06 15:59:21 dist-test-slave-tm1g krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031961, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:21.072767 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:21.060584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:36659 (local address 127.25.31.190:39565)
0806 15:59:21.060711 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:21.060715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:21.061481 (+ 766us) server_negotiation.cc:407] Connection header received
0806 15:59:21.062884 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:21.062887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:21.062936 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:21.063011 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:21.064092 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:21.064642 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:21.065742 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:21.065949 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:21.066024 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:21.068723 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:21.068727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:21.068730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:21.068752 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:21.070401 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:21.071061 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:21.071066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:21.071070 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:21.071173 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:21.071540 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:21.071545 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:21.071548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:21.071801 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:21.071929 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:21.072386 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 15:59:21.072525 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 15:59:21.073421 29824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:21.060875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:39565 (local address 127.25.31.130:36659)
0806 15:59:21.061311 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:21.061359 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:21.062690 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:21.063151 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:21.063158 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:21.063493 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:21.063950 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:21.063963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:21.064753 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:21.064755 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:21.065641 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:21.065650 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:21.066071 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:21.066073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:21.066206 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:21.066880 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:21.066905 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:21.068562 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:21.070537 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:21.070546 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:21.070563 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:21.070954 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:21.071298 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:21.071301 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:21.071302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:21.071443 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:21.071923 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:21.071928 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:21.072144 (+ 216us) client_negotiation.cc:769] Sending connection context
0806 15:59:21.072296 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 15:59:21.072542 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":103,"threads_started":1}
I20260806 15:59:21.074604 29822 heartbeater.cc:347] Connected to a master server at 127.25.31.190:39565
I20260806 15:59:21.074884 29822 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:21.075515 29822 heartbeater.cc:511] Master 127.25.31.190:39565 requested a full tablet report, sending...
I20260806 15:59:21.076623 29473 ts_manager.cc:194] Registered new tserver with Master: 283a58b05a3d49f1adca53548023b017 (127.25.31.130:42813)
I20260806 15:59:21.077189 29473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:36659
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 15:59:21.098565 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:39565
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:21.208295 29827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:21.208518 29827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:21.208559 29827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:21.212401 29827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:21.212446 29827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:21.212499 29827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:21.217720 29827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:39565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:21.218601 29827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:21.219485 29827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:21.228647 29835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:21.228603 29832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:21.228674 29827 server_base.cc:1034] running on GCE node
W20260806 15:59:21.228624 29833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:21.229243 29827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:21.229875 29827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:21.231211 29827 hybrid_clock.cc:648] HybridClock initialized: now 1786031961231181 us; error 38 us; skew 500 ppm
Aug 06 15:59:21 dist-test-slave-tm1g krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031961, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:21.233806 29827 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 15:59:21.234666 29827 webserver.cc:460] Webserver started at http://127.25.31.131:34211/ using document root <none> and password file <none>
I20260806 15:59:21.235188 29827 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:21.235277 29827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:21.235524 29827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:21.237418 29827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "27698a0825d6468fbea8390f3a3b9d00"
format_stamp: "Formatted at 2026-08-06 15:59:21 on dist-test-slave-tm1g"
server_key: "e46196875816d86e4fcaf2c0597fbcc5"
server_key_iv: "dfb0475ad694428158d2bae63c723eae"
server_key_version: "encryptionkey@0"
I20260806 15:59:21.237986 29827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "27698a0825d6468fbea8390f3a3b9d00"
format_stamp: "Formatted at 2026-08-06 15:59:21 on dist-test-slave-tm1g"
server_key: "e46196875816d86e4fcaf2c0597fbcc5"
server_key_iv: "dfb0475ad694428158d2bae63c723eae"
server_key_version: "encryptionkey@0"
I20260806 15:59:21.241688 29827 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 15:59:21.244134 29842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:21.245261 29827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:21.245406 29827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "27698a0825d6468fbea8390f3a3b9d00"
format_stamp: "Formatted at 2026-08-06 15:59:21 on dist-test-slave-tm1g"
server_key: "e46196875816d86e4fcaf2c0597fbcc5"
server_key_iv: "dfb0475ad694428158d2bae63c723eae"
server_key_version: "encryptionkey@0"
I20260806 15:59:21.245522 29827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:21.255556 29827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:21.258038 29827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:21.258227 29827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:21.258781 29827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:21.259557 29827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:21.259627 29827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:21.259714 29827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:21.259764 29827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:21.268247 29827 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:39493
I20260806 15:59:21.268268 29969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:39493 every 8 connection(s)
I20260806 15:59:21.269071 29827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 15:59:21.274569 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 29827
I20260806 15:59:21.274694 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 15:59:21.274978 25726 external_mini_cluster.cc:1428] Setting key ce4bbcad723cf24465e0d8ea735596ef
Aug 06 15:59:21 dist-test-slave-tm1g krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031961, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:21.282341 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:21.270793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:44413 (local address 127.25.31.190:39565)
0806 15:59:21.270928 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:21.270931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:21.271647 (+ 716us) server_negotiation.cc:407] Connection header received
0806 15:59:21.272979 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:21.272982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:21.273040 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:21.273149 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:21.274250 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:21.275108 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:21.276326 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:21.276590 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:21.276669 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:21.279159 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:21.279163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:21.279166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:21.279191 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:21.280377 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:21.280862 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:21.280866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:21.280868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:21.280950 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:21.281283 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:21.281287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:21.281289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:21.281466 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:21.281574 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:21.282038 (+ 464us) server_negotiation.cc:299] Negotiation successful
0806 15:59:21.282154 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 15:59:21.283043 29972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:21.271094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:39565 (local address 127.25.31.131:44413)
0806 15:59:21.271501 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:21.271542 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:21.272781 (+ 1239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:21.273315 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:21.273321 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:21.273665 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:21.274111 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:21.274123 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:21.275236 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:21.275239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:21.276212 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:21.276223 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:21.276739 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:21.276743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:21.276956 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:21.277545 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:21.277560 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:21.279000 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:21.280495 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:21.280501 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:21.280513 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:21.280758 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:21.281048 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:21.281052 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:21.281054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:21.281188 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:21.281571 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:21.281577 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:21.281817 (+ 240us) client_negotiation.cc:769] Sending connection context
0806 15:59:21.281993 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 15:59:21.282290 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":99,"threads_started":1}
I20260806 15:59:21.284102 29970 heartbeater.cc:347] Connected to a master server at 127.25.31.190:39565
I20260806 15:59:21.284330 29970 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:21.284837 29970 heartbeater.cc:511] Master 127.25.31.190:39565 requested a full tablet report, sending...
I20260806 15:59:21.285812 29473 ts_manager.cc:194] Registered new tserver with Master: 27698a0825d6468fbea8390f3a3b9d00 (127.25.31.131:39493)
I20260806 15:59:21.286424 29473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:44413
I20260806 15:59:21.288669 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 15:59:21 dist-test-slave-tm1g krb5kdc[29424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 15:59:21 dist-test-slave-tm1g krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.129@KRBTEST.COM
I20260806 15:59:21.314512 29977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:21.305195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.25.31.129:42259)
0806 15:59:21.305582 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:21.305591 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:21.305613 (+ 22us) server_negotiation.cc:407] Connection header received
0806 15:59:21.305663 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:21.305667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:21.305841 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:21.305961 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:21.306707 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:21.307478 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:21.308373 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:21.308735 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:21.308818 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:21.311140 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:21.311149 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:21.311161 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:21.311183 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:21.312857 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:21.313224 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:21.313228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:21.313229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:21.313335 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:21.313578 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:21.313581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:21.313582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:21.313802 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:21.313950 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:21.314121 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 15:59:21.314234 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":175,"threads_started":1}
Aug 06 15:59:21 dist-test-slave-tm1g krb5kdc[29424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:21.324900 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:21.317184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59868 (local address 127.25.31.190:39565)
0806 15:59:21.317323 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:21.317326 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:21.317359 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:59:21.317468 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:21.317470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:21.317515 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:21.317573 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:21.318159 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:21.318754 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:21.319469 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:21.319676 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:21.319746 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:21.321860 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:21.321863 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:21.321865 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:21.321887 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:21.323267 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:21.323669 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:21.323673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:21.323675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:21.323753 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:21.324019 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:21.324024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:21.324028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:21.324234 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:21.324370 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:21.324618 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 15:59:21.324761 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 15:59:21.325273 29473 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59868
I20260806 15:59:21.325840 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 29539
I20260806 15:59:21.332827 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 29683
I20260806 15:59:21.338596 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 29827
I20260806 15:59:21.344664 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 29437
2026-08-06T15:59:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2116 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:59:21 dist-test-slave-tm1g krb5kdc[29984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:59:21 dist-test-slave-tm1g krb5kdc[29984](info): set up 2 sockets
Aug 06 15:59:21 dist-test-slave-tm1g krb5kdc[29984](info): commencing operation
krb5kdc: starting...
W20260806 15:59:22.398824 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 15:59:22 dist-test-slave-tm1g krb5kdc[29984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:59:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 15:59:22.517125 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:35457
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:34717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:35457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:22.624912 29998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:22.625151 29998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:22.625193 29998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:22.629521 29998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:22.629588 29998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:22.629609 29998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:22.629626 29998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:22.629642 29998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:22.634918 29998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:34717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:35457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:35457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:22.635910 29998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:22.636893 29998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:22.645150 30003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:22.645150 30006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:22.645423 29998 server_base.cc:1034] running on GCE node
W20260806 15:59:22.645164 30004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:22.646080 29998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:22.647166 29998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:22.648348 29998 hybrid_clock.cc:648] HybridClock initialized: now 1786031962648326 us; error 49 us; skew 500 ppm
Aug 06 15:59:22 dist-test-slave-tm1g krb5kdc[29984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031962, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:22.651021 29998 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 15:59:22.651743 29998 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:22.651835 29998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:22.652070 29998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:22.654037 29998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "90c6556fc3624da08c2a7adf0b83675c"
format_stamp: "Formatted at 2026-08-06 15:59:22 on dist-test-slave-tm1g"
server_key: "a5db8f2adf31bc8df1300c1de99b212a"
server_key_iv: "d23c1722b21fdf136edc51c1acfca241"
server_key_version: "encryptionkey@0"
I20260806 15:59:22.654613 29998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "90c6556fc3624da08c2a7adf0b83675c"
format_stamp: "Formatted at 2026-08-06 15:59:22 on dist-test-slave-tm1g"
server_key: "a5db8f2adf31bc8df1300c1de99b212a"
server_key_iv: "d23c1722b21fdf136edc51c1acfca241"
server_key_version: "encryptionkey@0"
I20260806 15:59:22.658082 29998 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 15:59:22.660377 30013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:22.661629 29998 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:22.661777 29998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "90c6556fc3624da08c2a7adf0b83675c"
format_stamp: "Formatted at 2026-08-06 15:59:22 on dist-test-slave-tm1g"
server_key: "a5db8f2adf31bc8df1300c1de99b212a"
server_key_iv: "d23c1722b21fdf136edc51c1acfca241"
server_key_version: "encryptionkey@0"
I20260806 15:59:22.661892 29998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:22.677892 29998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:22.680789 29998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:22.680996 29998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:22.687916 29998 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:35457
I20260806 15:59:22.688200 30067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:35457 every 8 connection(s)
I20260806 15:59:22.688580 29998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:22.691416 30068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:22.692744 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 29998
I20260806 15:59:22.692898 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:22.693188 25726 external_mini_cluster.cc:1428] Setting key 8ff1a500f51b96a7db1a2637c3b10b00
I20260806 15:59:22.698233 30068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c: Bootstrap starting.
I20260806 15:59:22.700804 30068 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:22.701577 30068 log.cc:824] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c: Log is configured to *not* fsync() on all Append() calls
Aug 06 15:59:22 dist-test-slave-tm1g krb5kdc[29984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:22.703838 30068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c: No bootstrap required, opened a new log
I20260806 15:59:22.708760 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:22.694702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38626 (local address 127.25.31.190:35457)
0806 15:59:22.695136 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:22.695155 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:22.695197 (+ 42us) server_negotiation.cc:407] Connection header received
0806 15:59:22.696020 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:22.696043 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:22.696455 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:22.696814 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:22.697528 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:22.698963 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:22.699890 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:22.700542 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:22.700722 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:22.703560 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:22.703566 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:22.703581 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:22.703613 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:22.705576 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:22.706003 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:22.706012 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:22.706020 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:22.706170 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:22.706500 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:22.706505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:22.706510 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:22.707037 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:22.707221 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:22.707491 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 15:59:22.707784 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":113,"threads_started":1}
I20260806 15:59:22.708706 30068 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c6556fc3624da08c2a7adf0b83675c" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35457 } }
I20260806 15:59:22.708993 30068 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:22.709054 30068 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c6556fc3624da08c2a7adf0b83675c, State: Initialized, Role: FOLLOWER
I20260806 15:59:22.709532 30068 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c6556fc3624da08c2a7adf0b83675c" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35457 } }
I20260806 15:59:22.709686 30068 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:22.709762 30068 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:22.709865 30068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:22.711035 30068 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c6556fc3624da08c2a7adf0b83675c" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35457 } }
I20260806 15:59:22.711400 30068 leader_election.cc:302] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90c6556fc3624da08c2a7adf0b83675c; no voters:
I20260806 15:59:22.711702 30068 leader_election.cc:288] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:22.711840 30074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:22.712109 30074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [term 1 LEADER]: Becoming Leader. State: Replica: 90c6556fc3624da08c2a7adf0b83675c, State: Running, Role: LEADER
I20260806 15:59:22.712486 30074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c6556fc3624da08c2a7adf0b83675c" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35457 } }
I20260806 15:59:22.712934 30068 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:22.714172 30076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90c6556fc3624da08c2a7adf0b83675c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c6556fc3624da08c2a7adf0b83675c" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35457 } } }
I20260806 15:59:22.714349 30076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:22.714792 30084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:22.714753 30077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90c6556fc3624da08c2a7adf0b83675c. Latest consensus state: current_term: 1 leader_uuid: "90c6556fc3624da08c2a7adf0b83675c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90c6556fc3624da08c2a7adf0b83675c" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35457 } } }
I20260806 15:59:22.714952 30077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:22.718227 30084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:22.723127 30084 catalog_manager.cc:1347] Generated new cluster ID: dcbeaa20404e4d9eac0e38611c129040
I20260806 15:59:22.723192 30084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:22.735824 30084 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:22.736822 30084 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:22.741410 30084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c: Generated new TSK 0
I20260806 15:59:22.742026 30084 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 15:59:22.773932 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35457
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:34717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:22.884110 30096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:22.884400 30096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:22.884477 30096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:22.888450 30096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:22.888532 30096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:22.888621 30096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:22.893667 30096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:34717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:22.894677 30096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:22.895671 30096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:22.904774 30102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:22.904836 30096 server_base.cc:1034] running on GCE node
W20260806 15:59:22.904801 30104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:22.904785 30101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:22.905504 30096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:22.906150 30096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:22.907331 30096 hybrid_clock.cc:648] HybridClock initialized: now 1786031962907301 us; error 57 us; skew 500 ppm
Aug 06 15:59:22 dist-test-slave-tm1g krb5kdc[29984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031962, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:22.909996 30096 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 15:59:22.910751 30096 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:22.910847 30096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:22.911083 30096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:22.913054 30096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "4bda14292b9d4ec393ef18431d08177d"
format_stamp: "Formatted at 2026-08-06 15:59:22 on dist-test-slave-tm1g"
server_key: "4cf01a1751aac2972a35d0a35f71d99b"
server_key_iv: "d0d79f34b30b5c55f6109e129880075e"
server_key_version: "encryptionkey@0"
I20260806 15:59:22.913666 30096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "4bda14292b9d4ec393ef18431d08177d"
format_stamp: "Formatted at 2026-08-06 15:59:22 on dist-test-slave-tm1g"
server_key: "4cf01a1751aac2972a35d0a35f71d99b"
server_key_iv: "d0d79f34b30b5c55f6109e129880075e"
server_key_version: "encryptionkey@0"
I20260806 15:59:22.917309 30096 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 15:59:22.919628 30110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:22.920795 30096 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 15:59:22.920931 30096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "4bda14292b9d4ec393ef18431d08177d"
format_stamp: "Formatted at 2026-08-06 15:59:22 on dist-test-slave-tm1g"
server_key: "4cf01a1751aac2972a35d0a35f71d99b"
server_key_iv: "d0d79f34b30b5c55f6109e129880075e"
server_key_version: "encryptionkey@0"
I20260806 15:59:22.921044 30096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:22.969709 30096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:22.972594 30096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:22.972805 30096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:22.973421 30096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:22.974267 30096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:22.974370 30096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:22.974462 30096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:22.974511 30096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:22.983397 30096 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:41529
I20260806 15:59:22.983417 30225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:41529 every 8 connection(s)
I20260806 15:59:22.983978 30096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:22.990722 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 30096
I20260806 15:59:22.990825 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 15:59:22.991122 25726 external_mini_cluster.cc:1428] Setting key 66da303d7b80e8bd001ffa89755bf3b1
Aug 06 15:59:22 dist-test-slave-tm1g krb5kdc[29984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031962, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:22.997707 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:22.985677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:60423 (local address 127.25.31.190:35457)
0806 15:59:22.985809 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:22.985813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:22.986658 (+ 845us) server_negotiation.cc:407] Connection header received
0806 15:59:22.987941 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:22.987945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:22.988014 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:22.988118 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:22.989258 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:22.989806 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:22.991054 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:22.991352 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:22.991464 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:22.993887 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:22.993890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:22.993892 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:22.993915 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:22.995374 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:22.995997 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:22.996000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:22.996001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:22.996080 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:22.996510 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:22.996513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:22.996515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:22.996735 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:22.996830 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:22.997481 (+ 651us) server_negotiation.cc:299] Negotiation successful
0806 15:59:22.997589 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 15:59:22.998958 30229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:22.986022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:35457 (local address 127.25.31.129:60423)
0806 15:59:22.986501 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:22.986553 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:22.987733 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:22.988290 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:22.988298 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:22.988648 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:22.989106 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:22.989119 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:22.989936 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:22.989942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:22.990934 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:22.990949 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:22.991513 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:22.991517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:22.991684 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:22.992282 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:22.992298 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:22.993719 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:22.995512 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:22.995521 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:22.995538 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:22.995889 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:22.996185 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:22.996191 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:22.996195 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:22.996388 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:22.996857 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:22.996867 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:22.997204 (+ 337us) client_negotiation.cc:769] Sending connection context
0806 15:59:22.997425 (+ 221us) client_negotiation.cc:240] Negotiation successful
0806 15:59:22.997785 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":105,"threads_started":1}
I20260806 15:59:23.000232 30226 heartbeater.cc:347] Connected to a master server at 127.25.31.190:35457
I20260806 15:59:23.000489 30226 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:23.001085 30226 heartbeater.cc:511] Master 127.25.31.190:35457 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
I20260806 15:59:23.002777 30030 ts_manager.cc:194] Registered new tserver with Master: 4bda14292b9d4ec393ef18431d08177d (127.25.31.129:41529)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
I20260806 15:59:23.004211 30030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:60423
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 15:59:23.018817 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35457
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:34717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:23.129079 30232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:23.129366 30232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:23.129444 30232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:23.133405 30232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:23.133502 30232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:23.133596 30232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:23.138800 30232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:34717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:23.139843 30232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:23.141187 30232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:23.150611 30238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:23.150611 30237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:23.150615 30240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:23.150892 30232 server_base.cc:1034] running on GCE node
I20260806 15:59:23.151365 30232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:23.151994 30232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:23.153167 30232 hybrid_clock.cc:648] HybridClock initialized: now 1786031963153138 us; error 43 us; skew 500 ppm
Aug 06 15:59:23 dist-test-slave-tm1g krb5kdc[29984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031963, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:23.156050 30232 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 15:59:23.156759 30232 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:23.156854 30232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:23.157096 30232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:23.159214 30232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "088d0ce06fd54f4daadb47ec2708c8cb"
format_stamp: "Formatted at 2026-08-06 15:59:23 on dist-test-slave-tm1g"
server_key: "63355ac4579015950f7e59f1e5c02858"
server_key_iv: "8677d18aff003d45f40b129018097675"
server_key_version: "encryptionkey@0"
I20260806 15:59:23.159785 30232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "088d0ce06fd54f4daadb47ec2708c8cb"
format_stamp: "Formatted at 2026-08-06 15:59:23 on dist-test-slave-tm1g"
server_key: "63355ac4579015950f7e59f1e5c02858"
server_key_iv: "8677d18aff003d45f40b129018097675"
server_key_version: "encryptionkey@0"
I20260806 15:59:23.163434 30232 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 15:59:23.165972 30246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:23.167269 30232 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:23.167404 30232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "088d0ce06fd54f4daadb47ec2708c8cb"
format_stamp: "Formatted at 2026-08-06 15:59:23 on dist-test-slave-tm1g"
server_key: "63355ac4579015950f7e59f1e5c02858"
server_key_iv: "8677d18aff003d45f40b129018097675"
server_key_version: "encryptionkey@0"
I20260806 15:59:23.167518 30232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:23.188016 30232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:23.191123 30232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:23.191329 30232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:23.191906 30232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:23.192747 30232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:23.192821 30232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:23.192893 30232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:23.192936 30232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:23.201951 30232 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:35073
I20260806 15:59:23.201992 30367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:35073 every 8 connection(s)
I20260806 15:59:23.202625 30232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:23.205040 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 30232
I20260806 15:59:23.205176 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 15:59:23.205513 25726 external_mini_cluster.cc:1428] Setting key 491f70ee7dba3fbf255473dbcfea0272
Aug 06 15:59:23 dist-test-slave-tm1g krb5kdc[29984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031963, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
I20260806 15:59:23.217854 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:23.204325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:36195 (local address 127.25.31.190:35457)
0806 15:59:23.204465 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:23.204469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:23.205311 (+ 842us) server_negotiation.cc:407] Connection header received
0806 15:59:23.206793 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:23.206820 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:23.206869 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:23.206960 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:23.208557 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.209358 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.210907 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.211331 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.211464 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:23.213906 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:23.213910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:23.213912 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:23.213956 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:23.215757 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:23.216238 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:23.216242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:23.216244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:23.216333 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:23.216671 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:23.216675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:23.216677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:23.216954 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:23.217054 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:23.217553 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 15:59:23.217683 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
I20260806 15:59:23.218734 30370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:23.204691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:35457 (local address 127.25.31.130:36195)
0806 15:59:23.205150 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:23.205202 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:23.206584 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:23.207162 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:23.207172 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:23.207687 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:23.208394 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:23.208416 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.209515 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.209521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:23.210777 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:23.210788 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.211483 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.211486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:23.211624 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:23.212234 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:23.212249 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:23.213726 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:23.215859 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:23.215864 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:23.215878 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:23.216132 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:23.216429 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:23.216433 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:23.216435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:23.216576 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:23.217066 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:23.217074 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:23.217314 (+ 240us) client_negotiation.cc:769] Sending connection context
0806 15:59:23.217510 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 15:59:23.217822 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":134,"threads_started":1}
I20260806 15:59:23.219903 30368 heartbeater.cc:347] Connected to a master server at 127.25.31.190:35457
I20260806 15:59:23.220165 30368 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:23.220729 30368 heartbeater.cc:511] Master 127.25.31.190:35457 requested a full tablet report, sending...
I20260806 15:59:23.221895 30030 ts_manager.cc:194] Registered new tserver with Master: 088d0ce06fd54f4daadb47ec2708c8cb (127.25.31.130:35073)
I20260806 15:59:23.222554 30030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:36195
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 15:59:23.233209 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35457
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:34717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:23.343386 30373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:23.343621 30373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:23.343663 30373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:23.347968 30373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:23.348028 30373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:23.348086 30373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:23.353173 30373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:34717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:23.354058 30373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:23.355032 30373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:23.364311 30382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:23.364318 30379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:23.364498 30380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:23.364568 30373 server_base.cc:1034] running on GCE node
I20260806 15:59:23.364985 30373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:23.365638 30373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:23.366817 30373 hybrid_clock.cc:648] HybridClock initialized: now 1786031963366803 us; error 38 us; skew 500 ppm
Aug 06 15:59:23 dist-test-slave-tm1g krb5kdc[29984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031963, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:23.369602 30373 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 15:59:23.370388 30373 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:23.370491 30373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:23.370733 30373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:23.372748 30373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "1a323e5dd09548c09995736074244770"
format_stamp: "Formatted at 2026-08-06 15:59:23 on dist-test-slave-tm1g"
server_key: "2e7020722cce458745e4d07383052252"
server_key_iv: "0cc75afadfc3707a1edc40a21f731a1a"
server_key_version: "encryptionkey@0"
I20260806 15:59:23.373297 30373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "1a323e5dd09548c09995736074244770"
format_stamp: "Formatted at 2026-08-06 15:59:23 on dist-test-slave-tm1g"
server_key: "2e7020722cce458745e4d07383052252"
server_key_iv: "0cc75afadfc3707a1edc40a21f731a1a"
server_key_version: "encryptionkey@0"
I20260806 15:59:23.376894 30373 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 15:59:23.379237 30388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:23.380399 30373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:23.380541 30373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "1a323e5dd09548c09995736074244770"
format_stamp: "Formatted at 2026-08-06 15:59:23 on dist-test-slave-tm1g"
server_key: "2e7020722cce458745e4d07383052252"
server_key_iv: "0cc75afadfc3707a1edc40a21f731a1a"
server_key_version: "encryptionkey@0"
I20260806 15:59:23.380649 30373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:23.392158 30373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:23.394753 30373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:23.394948 30373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:23.395496 30373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:23.396306 30373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:23.396378 30373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:23.396446 30373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:23.396500 30373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:23.405735 30373 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:33621
I20260806 15:59:23.405762 30507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:33621 every 8 connection(s)
I20260806 15:59:23.406291 30373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 15:59:23.409195 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 30373
I20260806 15:59:23.409336 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 15:59:23.409629 25726 external_mini_cluster.cc:1428] Setting key 045a0a5806e46fad6fcefa59a92f0878
Aug 06 15:59:23 dist-test-slave-tm1g krb5kdc[29984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031963, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:23.419639 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:23.407939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:35183 (local address 127.25.31.190:35457)
0806 15:59:23.408072 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:23.408076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:23.408863 (+ 787us) server_negotiation.cc:407] Connection header received
0806 15:59:23.410470 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:23.410473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:23.410539 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:23.410645 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:23.411863 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.412398 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.413556 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.413782 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.413910 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:23.416300 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:23.416311 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:23.416313 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:23.416335 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:23.417575 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:23.418085 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:23.418093 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:23.418095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:23.418172 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:23.418576 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:23.418581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:23.418582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:23.418758 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:23.418868 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:23.419344 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 15:59:23.419467 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 15:59:23.420346 30510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:23.408277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:35457 (local address 127.25.31.131:35183)
0806 15:59:23.408718 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:23.408760 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:23.410222 (+ 1462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:23.410818 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:23.410825 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:23.411168 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:23.411726 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:23.411742 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.412549 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.412552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:23.413429 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:23.413438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.413931 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.413935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:23.414126 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:23.414666 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:23.414682 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:23.416145 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:23.417701 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:23.417707 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:23.417720 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:23.417966 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:23.418273 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:23.418276 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:23.418278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:23.418478 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:23.418867 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:23.418873 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:23.419110 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 15:59:23.419283 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 15:59:23.419585 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":133,"threads_started":1}
I20260806 15:59:23.421456 30508 heartbeater.cc:347] Connected to a master server at 127.25.31.190:35457
I20260806 15:59:23.421667 30508 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:23.422077 30508 heartbeater.cc:511] Master 127.25.31.190:35457 requested a full tablet report, sending...
I20260806 15:59:23.423105 30030 ts_manager.cc:194] Registered new tserver with Master: 1a323e5dd09548c09995736074244770 (127.25.31.131:33621)
I20260806 15:59:23.423736 30030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:35183
I20260806 15:59:23.434939 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:59:23.443785 30072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:23.436772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38628 (local address 127.25.31.190:35457)
0806 15:59:23.436883 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:23.436887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:23.437058 (+ 171us) server_negotiation.cc:407] Connection header received
0806 15:59:23.437141 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:23.437143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:23.437178 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:23.437241 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:23.437905 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.438485 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.439298 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.439506 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.439578 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:23.440886 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:23.440888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:23.440890 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:23.440906 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:23.442081 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:23.442533 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:23.442538 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:23.442540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:23.442624 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:23.442895 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:23.442899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:23.442901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:23.443118 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:23.443256 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:23.443509 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 15:59:23.443620 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 15:59:23.446450 30030 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:59:23.449434 30030 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:23.463896 30519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:23.457273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:41529 (local address 127.0.0.1:42170)
0806 15:59:23.458440 (+ 1167us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:23.458454 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:23.458524 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:23.459142 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:23.459145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:23.459179 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:23.459348 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:23.459355 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.460844 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.460848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:23.462293 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:23.462355 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.463381 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.463384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:23.463471 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:23.463500 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 15:59:23.463586 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 15:59:23.463722 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"mutex_wait_us":184,"spinlock_wait_cycles":2560,"thread_start_us":96,"threads_started":1}
I20260806 15:59:23.463907 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:23.458015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42170 (local address 127.25.31.129:41529)
0806 15:59:23.458671 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:23.458684 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:23.458706 (+ 22us) server_negotiation.cc:407] Connection header received
0806 15:59:23.458788 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:23.458795 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:23.459011 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:23.459165 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:23.459504 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.460685 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.462476 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.463246 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.463363 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:23.463504 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:23.463641 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 15:59:23.463737 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":86,"threads_started":1}
I20260806 15:59:23.464094 30520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:23.457556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:35073 (local address 127.0.0.1:38088)
0806 15:59:23.458077 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:23.458132 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:23.458264 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:23.459740 (+ 1476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:23.459743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:23.459760 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:23.459844 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:23.459850 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.460804 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.460811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:23.462612 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:23.462624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.463673 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.463676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:23.463752 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:23.463773 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 15:59:23.463865 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 15:59:23.463937 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":142,"spinlock_wait_cycles":6528,"thread_start_us":81,"threads_started":1}
I20260806 15:59:23.464447 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:23.457948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38088 (local address 127.25.31.130:35073)
0806 15:59:23.459361 (+ 1413us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:23.459375 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:23.459397 (+ 22us) server_negotiation.cc:407] Connection header received
0806 15:59:23.459472 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:23.459488 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:23.459614 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:23.459759 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:23.459933 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.460672 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.462744 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.463546 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.463660 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:23.463789 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:23.464181 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 15:59:23.464286 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":121,"threads_started":1}
I20260806 15:59:23.465248 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:23.457202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:33621 (local address 127.0.0.1:49402)
0806 15:59:23.458077 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:23.458131 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:23.458264 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:23.459128 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:23.459134 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:23.459156 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:23.459337 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:23.459345 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.460872 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.460875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:23.462613 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:23.462624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.464837 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.464844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:23.464946 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:23.464979 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 15:59:23.465029 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 15:59:23.465101 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":460,"spinlock_wait_cycles":6016,"thread_start_us":81,"threads_started":1}
I20260806 15:59:23.465987 30521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:23.457346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49402 (local address 127.25.31.131:33621)
0806 15:59:23.458594 (+ 1248us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:23.458609 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:23.458632 (+ 23us) server_negotiation.cc:407] Connection header received
0806 15:59:23.458708 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:23.458714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:23.458951 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:23.459128 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:23.459473 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.460685 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.463852 (+ 3167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.464696 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.465549 (+ 853us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:23.465687 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:23.465733 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 15:59:23.465827 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1119,"thread_start_us":163,"threads_started":1}
I20260806 15:59:23.467557 30299 tablet_service.cc:1467] Processing CreateTablet for tablet 37e6a50353964ea086eda6f35b60337a (DEFAULT_TABLE table=test-table [id=fea31d13d1674d5f83c6176d50dd63de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:23.467571 30158 tablet_service.cc:1467] Processing CreateTablet for tablet 37e6a50353964ea086eda6f35b60337a (DEFAULT_TABLE table=test-table [id=fea31d13d1674d5f83c6176d50dd63de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:23.468510 30299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37e6a50353964ea086eda6f35b60337a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:23.468608 30158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37e6a50353964ea086eda6f35b60337a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:23.468502 30438 tablet_service.cc:1467] Processing CreateTablet for tablet 37e6a50353964ea086eda6f35b60337a (DEFAULT_TABLE table=test-table [id=fea31d13d1674d5f83c6176d50dd63de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:23.469399 30438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37e6a50353964ea086eda6f35b60337a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:23.475068 30525 tablet_bootstrap.cc:492] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb: Bootstrap starting.
I20260806 15:59:23.476100 30526 tablet_bootstrap.cc:492] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d: Bootstrap starting.
I20260806 15:59:23.476241 30527 tablet_bootstrap.cc:492] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770: Bootstrap starting.
I20260806 15:59:23.477984 30526 tablet_bootstrap.cc:654] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:23.478433 30525 tablet_bootstrap.cc:654] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:23.478432 30527 tablet_bootstrap.cc:654] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:23.479166 30526 log.cc:824] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:23.479720 30525 log.cc:824] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:23.479720 30527 log.cc:824] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:23.481050 30526 tablet_bootstrap.cc:492] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d: No bootstrap required, opened a new log
I20260806 15:59:23.481276 30526 ts_tablet_manager.cc:1397] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 15:59:23.481623 30527 tablet_bootstrap.cc:492] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770: No bootstrap required, opened a new log
I20260806 15:59:23.481838 30527 ts_tablet_manager.cc:1397] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260806 15:59:23.482012 30525 tablet_bootstrap.cc:492] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb: No bootstrap required, opened a new log
I20260806 15:59:23.482223 30525 ts_tablet_manager.cc:1397] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260806 15:59:23.484637 30526 raft_consensus.cc:348] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a323e5dd09548c09995736074244770" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 33621 } } peers { permanent_uuid: "4bda14292b9d4ec393ef18431d08177d" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41529 } } peers { permanent_uuid: "088d0ce06fd54f4daadb47ec2708c8cb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 35073 } }
I20260806 15:59:23.484977 30526 raft_consensus.cc:374] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:23.485052 30526 raft_consensus.cc:729] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bda14292b9d4ec393ef18431d08177d, State: Initialized, Role: FOLLOWER
I20260806 15:59:23.485531 30526 consensus_queue.cc:260] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a323e5dd09548c09995736074244770" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 33621 } } peers { permanent_uuid: "4bda14292b9d4ec393ef18431d08177d" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41529 } } peers { permanent_uuid: "088d0ce06fd54f4daadb47ec2708c8cb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 35073 } }
I20260806 15:59:23.485682 30527 raft_consensus.cc:348] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a323e5dd09548c09995736074244770" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 33621 } } peers { permanent_uuid: "4bda14292b9d4ec393ef18431d08177d" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41529 } } peers { permanent_uuid: "088d0ce06fd54f4daadb47ec2708c8cb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 35073 } }
I20260806 15:59:23.486022 30527 raft_consensus.cc:374] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:23.485885 30525 raft_consensus.cc:348] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a323e5dd09548c09995736074244770" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 33621 } } peers { permanent_uuid: "4bda14292b9d4ec393ef18431d08177d" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41529 } } peers { permanent_uuid: "088d0ce06fd54f4daadb47ec2708c8cb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 35073 } }
I20260806 15:59:23.486119 30527 raft_consensus.cc:729] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a323e5dd09548c09995736074244770, State: Initialized, Role: FOLLOWER
I20260806 15:59:23.486173 30525 raft_consensus.cc:374] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:23.486176 30226 heartbeater.cc:503] Master 127.25.31.190:35457 was elected leader, sending a full tablet report...
I20260806 15:59:23.486274 30525 raft_consensus.cc:729] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 088d0ce06fd54f4daadb47ec2708c8cb, State: Initialized, Role: FOLLOWER
I20260806 15:59:23.486557 30526 ts_tablet_manager.cc:1428] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 15:59:23.486641 30527 consensus_queue.cc:260] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a323e5dd09548c09995736074244770" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 33621 } } peers { permanent_uuid: "4bda14292b9d4ec393ef18431d08177d" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41529 } } peers { permanent_uuid: "088d0ce06fd54f4daadb47ec2708c8cb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 35073 } }
I20260806 15:59:23.486733 30525 consensus_queue.cc:260] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a323e5dd09548c09995736074244770" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 33621 } } peers { permanent_uuid: "4bda14292b9d4ec393ef18431d08177d" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41529 } } peers { permanent_uuid: "088d0ce06fd54f4daadb47ec2708c8cb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 35073 } }
I20260806 15:59:23.487569 30508 heartbeater.cc:503] Master 127.25.31.190:35457 was elected leader, sending a full tablet report...
I20260806 15:59:23.487592 30368 heartbeater.cc:503] Master 127.25.31.190:35457 was elected leader, sending a full tablet report...
I20260806 15:59:23.487754 30527 ts_tablet_manager.cc:1428] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 15:59:23.489485 30525 ts_tablet_manager.cc:1428] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260806 15:59:23.657253 30509 tablet.cc:2369] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:59:23.703671 30369 tablet.cc:2369] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:59:23.735245 30228 tablet.cc:2369] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:59:23.892663 30531 raft_consensus.cc:482] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:23.892913 30531 raft_consensus.cc:504] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a323e5dd09548c09995736074244770" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 33621 } } peers { permanent_uuid: "4bda14292b9d4ec393ef18431d08177d" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41529 } } peers { permanent_uuid: "088d0ce06fd54f4daadb47ec2708c8cb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 35073 } }
I20260806 15:59:23.894166 30531 leader_election.cc:288] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a323e5dd09548c09995736074244770 (127.25.31.131:33621), 088d0ce06fd54f4daadb47ec2708c8cb (127.25.31.130:35073)
I20260806 15:59:23.898180 30521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:23.894531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:35627 (local address 127.25.31.131:33621)
0806 15:59:23.894688 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:23.894693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:23.894863 (+ 170us) server_negotiation.cc:407] Connection header received
0806 15:59:23.894992 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:23.894998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:23.895055 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:23.895131 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:23.895455 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.896028 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.897202 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.897604 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.897695 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:23.897775 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:23.897964 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 15:59:23.898041 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 15:59:23.898212 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:23.894612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:48451 (local address 127.25.31.130:35073)
0806 15:59:23.894737 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:23.894741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:23.895035 (+ 294us) server_negotiation.cc:407] Connection header received
0806 15:59:23.895177 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:23.895180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:23.895228 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:23.895316 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:23.895587 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.896063 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.897245 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.897639 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.897775 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:23.897827 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:23.898029 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 15:59:23.898109 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 15:59:23.898205 30534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:23.894417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:33621 (local address 127.25.31.129:35627)
0806 15:59:23.894760 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:23.894777 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:23.894884 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:23.895164 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:23.895167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:23.895186 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:23.895339 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:23.895346 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.896148 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.896155 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:23.897083 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:23.897093 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.897732 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.897738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:23.897828 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:23.897860 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 15:59:23.897927 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 15:59:23.898016 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":46,"thread_start_us":89,"threads_started":1}
I20260806 15:59:23.898221 30535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:23.894498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:35073 (local address 127.25.31.129:48451)
0806 15:59:23.894909 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:23.894941 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:23.895050 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:23.895336 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:23.895339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:23.895353 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:23.895476 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:23.895483 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.896204 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.896207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:23.897090 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:23.897099 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:23.897792 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:23.897796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:23.897884 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:23.897901 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 15:59:23.897995 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 15:59:23.898067 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":24,"thread_start_us":100,"threads_started":1}
I20260806 15:59:23.898907 30461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e6a50353964ea086eda6f35b60337a" candidate_uuid: "4bda14292b9d4ec393ef18431d08177d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a323e5dd09548c09995736074244770" is_pre_election: true
I20260806 15:59:23.899077 30320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e6a50353964ea086eda6f35b60337a" candidate_uuid: "4bda14292b9d4ec393ef18431d08177d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "088d0ce06fd54f4daadb47ec2708c8cb" is_pre_election: true
I20260806 15:59:23.899233 30461 raft_consensus.cc:2454] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bda14292b9d4ec393ef18431d08177d in term 0.
I20260806 15:59:23.899389 30320 raft_consensus.cc:2454] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bda14292b9d4ec393ef18431d08177d in term 0.
I20260806 15:59:23.899777 30111 leader_election.cc:302] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 088d0ce06fd54f4daadb47ec2708c8cb, 1a323e5dd09548c09995736074244770, 4bda14292b9d4ec393ef18431d08177d; no voters:
I20260806 15:59:23.900058 30531 raft_consensus.cc:2781] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:23.900146 30531 raft_consensus.cc:482] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:23.900188 30531 raft_consensus.cc:3037] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:23.901305 30531 raft_consensus.cc:504] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a323e5dd09548c09995736074244770" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 33621 } } peers { permanent_uuid: "4bda14292b9d4ec393ef18431d08177d" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41529 } } peers { permanent_uuid: "088d0ce06fd54f4daadb47ec2708c8cb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 35073 } }
I20260806 15:59:23.901670 30531 leader_election.cc:288] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [CANDIDATE]: Term 1 election: Requested vote from peers 1a323e5dd09548c09995736074244770 (127.25.31.131:33621), 088d0ce06fd54f4daadb47ec2708c8cb (127.25.31.130:35073)
I20260806 15:59:23.902040 30320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e6a50353964ea086eda6f35b60337a" candidate_uuid: "4bda14292b9d4ec393ef18431d08177d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "088d0ce06fd54f4daadb47ec2708c8cb"
I20260806 15:59:23.902031 30461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37e6a50353964ea086eda6f35b60337a" candidate_uuid: "4bda14292b9d4ec393ef18431d08177d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a323e5dd09548c09995736074244770"
I20260806 15:59:23.902173 30320 raft_consensus.cc:3037] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:23.902181 30461 raft_consensus.cc:3037] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:23.903226 30320 raft_consensus.cc:2454] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bda14292b9d4ec393ef18431d08177d in term 1.
I20260806 15:59:23.903249 30461 raft_consensus.cc:2454] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bda14292b9d4ec393ef18431d08177d in term 1.
I20260806 15:59:23.903560 30111 leader_election.cc:302] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a323e5dd09548c09995736074244770, 4bda14292b9d4ec393ef18431d08177d; no voters:
I20260806 15:59:23.903759 30531 raft_consensus.cc:2781] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:23.903995 30531 raft_consensus.cc:686] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [term 1 LEADER]: Becoming Leader. State: Replica: 4bda14292b9d4ec393ef18431d08177d, State: Running, Role: LEADER
I20260806 15:59:23.904315 30531 consensus_queue.cc:237] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a323e5dd09548c09995736074244770" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 33621 } } peers { permanent_uuid: "4bda14292b9d4ec393ef18431d08177d" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41529 } } peers { permanent_uuid: "088d0ce06fd54f4daadb47ec2708c8cb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 35073 } }
I20260806 15:59:23.907488 30029 catalog_manager.cc:5495] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bda14292b9d4ec393ef18431d08177d (127.25.31.129). New cstate: current_term: 1 leader_uuid: "4bda14292b9d4ec393ef18431d08177d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a323e5dd09548c09995736074244770" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 33621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bda14292b9d4ec393ef18431d08177d" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "088d0ce06fd54f4daadb47ec2708c8cb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 35073 } health_report { overall_health: UNKNOWN } } }
I20260806 15:59:24.026719 30541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.022907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42184 (local address 127.25.31.129:41529)
0806 15:59:24.023180 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:24.023184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:24.023200 (+ 16us) server_negotiation.cc:407] Connection header received
0806 15:59:24.023371 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:24.023374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:24.023441 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:24.023538 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:24.023822 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:24.024375 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:24.025293 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:24.025533 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:24.025609 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:24.025885 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:24.026293 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:24.026403 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:24.026464 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 15:59:24.026512 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":94,"threads_started":1}
I20260806 15:59:24.032158 30461 raft_consensus.cc:1261] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770 [term 1 FOLLOWER]: Refusing update from remote peer 4bda14292b9d4ec393ef18431d08177d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:24.032181 30320 raft_consensus.cc:1261] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb [term 1 FOLLOWER]: Refusing update from remote peer 4bda14292b9d4ec393ef18431d08177d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:24.032827 30536 consensus_queue.cc:1035] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a323e5dd09548c09995736074244770" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 33621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:24.033002 30531 consensus_queue.cc:1035] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [LEADER]: Connected to new peer: Peer: permanent_uuid: "088d0ce06fd54f4daadb47ec2708c8cb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 35073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:24.043057 30547 mvcc.cc:205] Tried to move back new op lower bound from 7315586924668952576 to 7315586924154331136. Current Snapshot: MvccSnapshot[applied={T|T < 7315586924668952576 or (T in {7315586924668952576})}]
I20260806 15:59:24.048550 30028 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:24.048724 30028 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:24.051190 30028 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90c6556fc3624da08c2a7adf0b83675c: Sending DeleteTablet for 3 replicas of tablet 37e6a50353964ea086eda6f35b60337a
I20260806 15:59:24.051929 30438 tablet_service.cc:1514] Processing DeleteTablet for tablet 37e6a50353964ea086eda6f35b60337a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:24 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:49402
I20260806 15:59:24.052078 30158 tablet_service.cc:1514] Processing DeleteTablet for tablet 37e6a50353964ea086eda6f35b60337a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:24 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:42170
I20260806 15:59:24.052145 30299 tablet_service.cc:1514] Processing DeleteTablet for tablet 37e6a50353964ea086eda6f35b60337a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:24 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:38088
I20260806 15:59:24.052441 30556 tablet_replica.cc:330] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770: stopping tablet replica
I20260806 15:59:24.052646 30557 tablet_replica.cc:330] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d: stopping tablet replica
I20260806 15:59:24.052794 30556 raft_consensus.cc:2229] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:24.052852 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 30096
I20260806 15:59:24.052919 30557 raft_consensus.cc:2229] T 37e6a50353964ea086eda6f35b60337a P 4bda14292b9d4ec393ef18431d08177d [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:59:24.053115 30556 raft_consensus.cc:2258] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:24.054670 30556 ts_tablet_manager.cc:1905] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:24.055486 30558 tablet_replica.cc:330] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb: stopping tablet replica
I20260806 15:59:24.055759 30558 raft_consensus.cc:2229] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:24.056054 30558 raft_consensus.cc:2258] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:24.057312 30558 ts_tablet_manager.cc:1905] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:24.057637 30556 ts_tablet_manager.cc:1918] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:24.057745 30556 log.cc:1196] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/37e6a50353964ea086eda6f35b60337a
I20260806 15:59:24.058094 30556 ts_tablet_manager.cc:1939] T 37e6a50353964ea086eda6f35b60337a P 1a323e5dd09548c09995736074244770: Deleting consensus metadata
I20260806 15:59:24.059368 30014 catalog_manager.cc:4841] TS 1a323e5dd09548c09995736074244770 (127.25.31.131:33621): tablet 37e6a50353964ea086eda6f35b60337a (table test-table [id=fea31d13d1674d5f83c6176d50dd63de]) successfully deleted
I20260806 15:59:24.059988 30558 ts_tablet_manager.cc:1918] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:24.060109 30558 log.cc:1196] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/37e6a50353964ea086eda6f35b60337a
I20260806 15:59:24.060472 30558 ts_tablet_manager.cc:1939] T 37e6a50353964ea086eda6f35b60337a P 088d0ce06fd54f4daadb47ec2708c8cb: Deleting consensus metadata
W20260806 15:59:24.060549 30017 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:41529 (error 108)
I20260806 15:59:24.060786 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 30232
I20260806 15:59:24.061947 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.061352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:41529 (local address 127.0.0.1:42198)
0806 15:59:24.061797 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:24.061860 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:41529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":56,"threads_started":1}
W20260806 15:59:24.062122 30017 catalog_manager.cc:4584] TS 4bda14292b9d4ec393ef18431d08177d (127.25.31.129:41529): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37e6a50353964ea086eda6f35b60337a: Network error: Client connection negotiation failed: client connection to 127.25.31.129:41529: connect: Connection refused (error 111)
I20260806 15:59:24.067938 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 30373
I20260806 15:59:24.068051 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.067771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:35073 (local address 127.0.0.1:38094)
0806 15:59:24.067907 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:24.067967 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.130:35073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 15:59:24.074224 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 29998
2026-08-06T15:59:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2730 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T15:59:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:24Z Disabled control of system clock
I20260806 15:59:24.105264 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:37947
--webserver_interface=127.25.31.190
--webserver_port=0
--builtin_ntp_servers=127.25.31.148:36815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:37947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 15:59:24.214004 30565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:24.214255 30565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:24.214293 30565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:24.218286 30565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:24.218361 30565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:24.218418 30565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:24.218452 30565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:24.218482 30565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:24.223306 30565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:36815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:37947
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:37947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:24.224295 30565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:24.225620 30565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:24.233886 30574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:24.234045 30565 server_base.cc:1034] running on GCE node
W20260806 15:59:24.233887 30571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:24.234256 30570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:24.234854 30565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:24.235904 30565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:24.237113 30565 hybrid_clock.cc:648] HybridClock initialized: now 1786031964237092 us; error 43 us; skew 500 ppm
I20260806 15:59:24.250844 30565 webserver.cc:460] Webserver started at http://127.25.31.190:45471/ using document root <none> and password file <none>
I20260806 15:59:24.251461 30565 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:24.251560 30565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:24.251807 30565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:24.253906 30565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "a2a917895426459aa6aaa9fdb795898d"
format_stamp: "Formatted at 2026-08-06 15:59:24 on dist-test-slave-tm1g"
server_key: "c2629b4f0b8b88dcd3a5fffec0726776"
server_key_iv: "1aff7ecd39fa520e2b59c681ddb20bd8"
server_key_version: "encryptionkey@0"
I20260806 15:59:24.254551 30565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "a2a917895426459aa6aaa9fdb795898d"
format_stamp: "Formatted at 2026-08-06 15:59:24 on dist-test-slave-tm1g"
server_key: "c2629b4f0b8b88dcd3a5fffec0726776"
server_key_iv: "1aff7ecd39fa520e2b59c681ddb20bd8"
server_key_version: "encryptionkey@0"
I20260806 15:59:24.258150 30565 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 15:59:24.260638 30580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:24.261821 30565 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:24.261982 30565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "a2a917895426459aa6aaa9fdb795898d"
format_stamp: "Formatted at 2026-08-06 15:59:24 on dist-test-slave-tm1g"
server_key: "c2629b4f0b8b88dcd3a5fffec0726776"
server_key_iv: "1aff7ecd39fa520e2b59c681ddb20bd8"
server_key_version: "encryptionkey@0"
I20260806 15:59:24.262099 30565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:24.266693 30565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:24.267304 30565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:24.267498 30565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:24.274395 30565 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:37947
I20260806 15:59:24.274664 30634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:37947 every 8 connection(s)
I20260806 15:59:24.275323 30565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:24.278493 30635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:24.280643 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 30565
I20260806 15:59:24.280797 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:24.281075 25726 external_mini_cluster.cc:1428] Setting key e848b16521a1a2f6f98fd5d4ea584d5c
I20260806 15:59:24.284057 30635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d: Bootstrap starting.
I20260806 15:59:24.286458 30635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:24.286656 30638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.282456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60698 (local address 127.25.31.190:37947)
0806 15:59:24.283112 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:24.283135 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:24.283187 (+ 52us) server_negotiation.cc:407] Connection header received
0806 15:59:24.283809 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:24.283825 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:24.284186 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:24.284530 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:24.285343 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:24.285348 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:24.285358 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:24.285370 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:24.285425 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:24.285441 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:24.285592 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:24.285664 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 15:59:24.285893 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":131,"threads_started":1}
I20260806 15:59:24.287408 30635 log.cc:824] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:24.289302 30635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d: No bootstrap required, opened a new log
I20260806 15:59:24.293053 30635 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a917895426459aa6aaa9fdb795898d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37947 } }
I20260806 15:59:24.293286 30635 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:24.293362 30635 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a917895426459aa6aaa9fdb795898d, State: Initialized, Role: FOLLOWER
I20260806 15:59:24.293766 30635 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a917895426459aa6aaa9fdb795898d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37947 } }
I20260806 15:59:24.293932 30635 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:24.294013 30635 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:24.294127 30635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:24.295054 30635 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a917895426459aa6aaa9fdb795898d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37947 } }
I20260806 15:59:24.295384 30635 leader_election.cc:302] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2a917895426459aa6aaa9fdb795898d; no voters:
I20260806 15:59:24.295674 30635 leader_election.cc:288] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:24.295809 30640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:24.296056 30640 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [term 1 LEADER]: Becoming Leader. State: Replica: a2a917895426459aa6aaa9fdb795898d, State: Running, Role: LEADER
I20260806 15:59:24.296409 30640 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a917895426459aa6aaa9fdb795898d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37947 } }
I20260806 15:59:24.296855 30635 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:24.298167 30641 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2a917895426459aa6aaa9fdb795898d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a917895426459aa6aaa9fdb795898d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37947 } } }
I20260806 15:59:24.298295 30641 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:24.298624 30642 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2a917895426459aa6aaa9fdb795898d. Latest consensus state: current_term: 1 leader_uuid: "a2a917895426459aa6aaa9fdb795898d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2a917895426459aa6aaa9fdb795898d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37947 } } }
I20260806 15:59:24.298719 30642 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:24.298851 30650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:24.301528 30650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:24.306416 30650 catalog_manager.cc:1347] Generated new cluster ID: 1f65e0a836f742e6891d705f68815995
I20260806 15:59:24.306478 30650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:24.318784 30650 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:24.319113 30650 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:24.325141 30650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d: Generated new TSK 0
I20260806 15:59:24.325775 30650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:59:24.333101 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37947
--builtin_ntp_servers=127.25.31.148:36815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 15:59:24.450683 30660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:24.450929 30660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:24.450969 30660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:24.455039 30660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:24.455094 30660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:24.455149 30660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:24.459980 30660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:36815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:24.461041 30660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:24.462446 30660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:24.472028 30667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:24.472086 30660 server_base.cc:1034] running on GCE node
W20260806 15:59:24.472112 30665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:24.472213 30669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:24.472659 30660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:24.473284 30660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:24.474493 30660 hybrid_clock.cc:648] HybridClock initialized: now 1786031964474474 us; error 28 us; skew 500 ppm
I20260806 15:59:24.476483 30660 webserver.cc:460] Webserver started at http://127.25.31.129:38761/ using document root <none> and password file <none>
I20260806 15:59:24.477100 30660 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:24.477202 30660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:24.477470 30660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:24.479678 30660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "66194b488877459d98a0363508d56fae"
format_stamp: "Formatted at 2026-08-06 15:59:24 on dist-test-slave-tm1g"
server_key: "9e0d7fbe86aa51c3acdede739f157490"
server_key_iv: "60a66ae402740005b274cfb700679be5"
server_key_version: "encryptionkey@0"
I20260806 15:59:24.480273 30660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "66194b488877459d98a0363508d56fae"
format_stamp: "Formatted at 2026-08-06 15:59:24 on dist-test-slave-tm1g"
server_key: "9e0d7fbe86aa51c3acdede739f157490"
server_key_iv: "60a66ae402740005b274cfb700679be5"
server_key_version: "encryptionkey@0"
I20260806 15:59:24.484108 30660 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 15:59:24.486744 30675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:24.487957 30660 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:24.488098 30660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "66194b488877459d98a0363508d56fae"
format_stamp: "Formatted at 2026-08-06 15:59:24 on dist-test-slave-tm1g"
server_key: "9e0d7fbe86aa51c3acdede739f157490"
server_key_iv: "60a66ae402740005b274cfb700679be5"
server_key_version: "encryptionkey@0"
I20260806 15:59:24.488214 30660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:24.492687 30660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:24.493283 30660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:24.493475 30660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:24.494009 30660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:24.494989 30660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:24.495060 30660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:24.495126 30660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:24.495168 30660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:24.504252 30660 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:34349
I20260806 15:59:24.504281 30794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:34349 every 8 connection(s)
I20260806 15:59:24.505194 30660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:24.510429 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 30660
I20260806 15:59:24.510546 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 15:59:24.510885 25726 external_mini_cluster.cc:1428] Setting key b4275594ac807be986f4f459b53f5eba
I20260806 15:59:24.512090 30638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.507202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:39051 (local address 127.25.31.190:37947)
0806 15:59:24.507380 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:24.507385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:24.508496 (+ 1111us) server_negotiation.cc:407] Connection header received
0806 15:59:24.509947 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:24.509950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:24.509990 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:24.510072 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:24.511546 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:24.511550 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:24.511553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:24.511575 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:24.511597 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:24.511606 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:24.511710 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:24.511891 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 15:59:24.511943 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 15:59:24.512910 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37947
--builtin_ntp_servers=127.25.31.148:36815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 15:59:24.513070 30797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.507669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37947 (local address 127.25.31.129:39051)
0806 15:59:24.508342 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:24.508383 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:24.509716 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:24.510309 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:24.510353 (+ 44us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:24.510481 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:24.511201 (+ 720us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:59:24.511203 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:24.511247 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:59:24.511355 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:24.511708 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:24.511720 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:24.511730 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 15:59:24.511928 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 15:59:24.512189 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":114,"threads_started":1}
I20260806 15:59:24.514430 30795 heartbeater.cc:347] Connected to a master server at 127.25.31.190:37947
I20260806 15:59:24.514678 30795 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:24.515122 30795 heartbeater.cc:511] Master 127.25.31.190:37947 requested a full tablet report, sending...
I20260806 15:59:24.516757 30597 ts_manager.cc:194] Registered new tserver with Master: 66194b488877459d98a0363508d56fae (127.25.31.129:34349)
W20260806 15:59:24.628137 30798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:24.628417 30798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:24.628459 30798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:24.632655 30798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:24.632714 30798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:24.632818 30798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:24.637632 30798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:36815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:24.638798 30798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:24.640439 30798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:24.650247 30804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:24.650255 30803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:24.650255 30806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:24.650449 30798 server_base.cc:1034] running on GCE node
I20260806 15:59:24.651026 30798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:24.651693 30798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:24.652884 30798 hybrid_clock.cc:648] HybridClock initialized: now 1786031964652849 us; error 45 us; skew 500 ppm
I20260806 15:59:24.654752 30798 webserver.cc:460] Webserver started at http://127.25.31.130:36931/ using document root <none> and password file <none>
I20260806 15:59:24.655351 30798 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:24.655450 30798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:24.655690 30798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:24.657783 30798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "fc73268a8a8a4dccbb947ac7b8f0507a"
format_stamp: "Formatted at 2026-08-06 15:59:24 on dist-test-slave-tm1g"
server_key: "977adf37a530f17eafcf41801d25f83d"
server_key_iv: "edbcf810ddf09492c1efd3e1b4463284"
server_key_version: "encryptionkey@0"
I20260806 15:59:24.658438 30798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "fc73268a8a8a4dccbb947ac7b8f0507a"
format_stamp: "Formatted at 2026-08-06 15:59:24 on dist-test-slave-tm1g"
server_key: "977adf37a530f17eafcf41801d25f83d"
server_key_iv: "edbcf810ddf09492c1efd3e1b4463284"
server_key_version: "encryptionkey@0"
I20260806 15:59:24.662241 30798 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 15:59:24.664908 30814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:24.666355 30798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260806 15:59:24.666498 30798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "fc73268a8a8a4dccbb947ac7b8f0507a"
format_stamp: "Formatted at 2026-08-06 15:59:24 on dist-test-slave-tm1g"
server_key: "977adf37a530f17eafcf41801d25f83d"
server_key_iv: "edbcf810ddf09492c1efd3e1b4463284"
server_key_version: "encryptionkey@0"
I20260806 15:59:24.666632 30798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:24.672816 30798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:24.673465 30798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:24.673657 30798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:24.674217 30798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:24.675174 30798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:24.675248 30798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:24.675312 30798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:24.675362 30798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:24.684072 30798 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:39961
I20260806 15:59:24.684093 30930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:39961 every 8 connection(s)
I20260806 15:59:24.685004 30798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:24.690078 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 30798
I20260806 15:59:24.690228 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 15:59:24.690573 25726 external_mini_cluster.cc:1428] Setting key bd50f51d8f1adb5485e56baa370fd217
I20260806 15:59:24.691425 30638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.686897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:46431 (local address 127.25.31.190:37947)
0806 15:59:24.687038 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:24.687042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:24.688068 (+ 1026us) server_negotiation.cc:407] Connection header received
0806 15:59:24.689416 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:24.689419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:24.689470 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:24.689582 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:24.690798 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:24.690802 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:24.690805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:24.690822 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:24.690852 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:24.690864 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:24.690986 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:24.691222 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 15:59:24.691287 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 15:59:24.692440 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.687254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37947 (local address 127.25.31.130:46431)
0806 15:59:24.687904 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:24.687943 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:24.689191 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:24.689749 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:24.689756 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:24.689844 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:24.690487 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:59:24.690488 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:24.690531 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:59:24.690598 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:24.691038 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:24.691050 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:24.691058 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 15:59:24.691209 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 15:59:24.691480 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":123,"threads_started":1}
I20260806 15:59:24.692953 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37947
--builtin_ntp_servers=127.25.31.148:36815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 15:59:24.693758 30931 heartbeater.cc:347] Connected to a master server at 127.25.31.190:37947
I20260806 15:59:24.694023 30931 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:24.694546 30931 heartbeater.cc:511] Master 127.25.31.190:37947 requested a full tablet report, sending...
I20260806 15:59:24.695716 30597 ts_manager.cc:194] Registered new tserver with Master: fc73268a8a8a4dccbb947ac7b8f0507a (127.25.31.130:39961)
W20260806 15:59:24.808758 30934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:24.809036 30934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:24.809118 30934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:24.813159 30934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:24.813241 30934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:24.813334 30934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:24.817977 30934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:36815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:24.819037 30934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:24.820472 30934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:24.829993 30942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:24.830039 30939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:24.830024 30940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:24.830242 30934 server_base.cc:1034] running on GCE node
I20260806 15:59:24.830646 30934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:24.831312 30934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:24.832468 30934 hybrid_clock.cc:648] HybridClock initialized: now 1786031964832435 us; error 45 us; skew 500 ppm
I20260806 15:59:24.834239 30934 webserver.cc:460] Webserver started at http://127.25.31.131:36357/ using document root <none> and password file <none>
I20260806 15:59:24.834841 30934 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:24.834904 30934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:24.835150 30934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:24.837157 30934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "7d607ed3c17e4d1cb69eadad32643083"
format_stamp: "Formatted at 2026-08-06 15:59:24 on dist-test-slave-tm1g"
server_key: "7dfa25826564510d03a791f62821e16b"
server_key_iv: "9fbcb806e7ef3d3b1992e43db4176307"
server_key_version: "encryptionkey@0"
I20260806 15:59:24.837718 30934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "7d607ed3c17e4d1cb69eadad32643083"
format_stamp: "Formatted at 2026-08-06 15:59:24 on dist-test-slave-tm1g"
server_key: "7dfa25826564510d03a791f62821e16b"
server_key_iv: "9fbcb806e7ef3d3b1992e43db4176307"
server_key_version: "encryptionkey@0"
I20260806 15:59:24.841415 30934 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 15:59:24.843880 30948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:24.845005 30934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 15:59:24.845134 30934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "7d607ed3c17e4d1cb69eadad32643083"
format_stamp: "Formatted at 2026-08-06 15:59:24 on dist-test-slave-tm1g"
server_key: "7dfa25826564510d03a791f62821e16b"
server_key_iv: "9fbcb806e7ef3d3b1992e43db4176307"
server_key_version: "encryptionkey@0"
I20260806 15:59:24.845237 30934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:24.849864 30934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:24.850555 30934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:24.850737 30934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:24.851266 30934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:24.852131 30934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:24.852176 30934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:24.852236 30934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:24.852270 30934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:24.860620 30934 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:39467
I20260806 15:59:24.860656 31069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:39467 every 8 connection(s)
I20260806 15:59:24.861472 30934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 15:59:24.867204 30638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.863234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:46835 (local address 127.25.31.190:37947)
0806 15:59:24.863366 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:24.863370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:24.864346 (+ 976us) server_negotiation.cc:407] Connection header received
0806 15:59:24.865635 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:24.865637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:24.865674 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:24.865748 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:24.866881 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:24.866884 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:24.866885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:24.866896 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:24.866916 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:24.866923 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:24.866999 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:24.867068 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 15:59:24.867101 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 15:59:24.868068 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.863587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37947 (local address 127.25.31.131:46835)
0806 15:59:24.864194 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:24.864232 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:24.865451 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:24.865869 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:24.865875 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:24.865961 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:24.866651 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:59:24.866652 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:24.866689 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:59:24.866751 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:24.866981 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:24.866986 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:24.866989 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:59:24.867106 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 15:59:24.867319 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":112,"threads_started":1}
I20260806 15:59:24.869259 31070 heartbeater.cc:347] Connected to a master server at 127.25.31.190:37947
I20260806 15:59:24.869490 31070 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:24.869889 31070 heartbeater.cc:511] Master 127.25.31.190:37947 requested a full tablet report, sending...
I20260806 15:59:24.870724 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 30934
I20260806 15:59:24.870860 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 15:59:24.871031 30597 ts_manager.cc:194] Registered new tserver with Master: 7d607ed3c17e4d1cb69eadad32643083 (127.25.31.131:39467)
I20260806 15:59:24.871199 25726 external_mini_cluster.cc:1428] Setting key 57d00fa84f4e7b27298dbbdc020bcb41
I20260806 15:59:24.873224 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:59:24.876524 30638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.875096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60706 (local address 127.25.31.190:37947)
0806 15:59:24.875211 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:24.875215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:24.875415 (+ 200us) server_negotiation.cc:407] Connection header received
0806 15:59:24.875506 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:24.875509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:24.875538 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:24.875599 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:24.876260 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:24.876262 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:24.876264 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:24.876272 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:24.876286 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:24.876292 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:24.876353 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:24.876394 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 15:59:24.876423 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 15:59:24.879077 30597 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:59:24.881942 30597 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:24.893185 31082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.889918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:39467 (local address 127.0.0.1:50844)
0806 15:59:24.890665 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:24.890683 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:24.890760 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:24.891588 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:24.891595 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:24.891671 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:24.892324 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:59:24.892328 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:24.892351 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:59:24.892418 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:24.892835 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:24.892844 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:24.892850 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 15:59:24.892957 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 15:59:24.893034 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":109,"threads_started":1}
I20260806 15:59:24.893195 31085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.890583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50844 (local address 127.25.31.131:39467)
0806 15:59:24.890904 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:24.890916 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:24.890938 (+ 22us) server_negotiation.cc:407] Connection header received
0806 15:59:24.890988 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:24.890992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:24.891148 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:24.891280 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:24.892633 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:24.892643 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:24.892657 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:24.892672 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:24.892718 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:24.892741 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:24.892861 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:24.892968 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 15:59:24.893038 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":107,"threads_started":1}
I20260806 15:59:24.893489 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.889911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40636 (local address 127.25.31.129:34349)
0806 15:59:24.890469 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:24.890484 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:24.890513 (+ 29us) server_negotiation.cc:407] Connection header received
0806 15:59:24.890566 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:24.890572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:24.890742 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:24.890883 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:24.892678 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:24.892684 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:24.892695 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:24.892708 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:24.892757 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:24.892791 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:24.892911 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:24.893068 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 15:59:24.893137 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":132,"threads_started":1}
I20260806 15:59:24.893226 31084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.889721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:34349 (local address 127.0.0.1:40636)
0806 15:59:24.890266 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:24.890339 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:24.890436 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:24.891588 (+ 1152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:24.891595 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:24.891671 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:24.892392 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:59:24.892394 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:24.892402 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:59:24.892418 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:24.892960 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:24.892963 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:24.892967 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:59:24.893049 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 15:59:24.893087 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":75,"threads_started":1}
I20260806 15:59:24.894310 31086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.890900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:39961 (local address 127.0.0.1:36372)
0806 15:59:24.891625 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:24.891639 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:24.891719 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:24.892188 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:24.892191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:24.892205 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:24.892558 (+ 353us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:59:24.892560 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:24.892568 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:59:24.892587 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:24.893349 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:24.893352 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:24.893356 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:59:24.894165 (+ 809us) client_negotiation.cc:240] Negotiation successful
0806 15:59:24.894201 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":86,"threads_started":1}
I20260806 15:59:24.894763 31087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:24.890958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36372 (local address 127.25.31.130:39961)
0806 15:59:24.891452 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:24.891466 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:24.891713 (+ 247us) server_negotiation.cc:407] Connection header received
0806 15:59:24.891805 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:24.891816 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:24.891987 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:24.892113 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:24.892791 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:24.892796 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:24.892806 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:24.892819 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:24.892857 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:24.892881 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:24.892979 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:24.894548 (+ 1569us) server_negotiation.cc:299] Negotiation successful
0806 15:59:24.894619 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":137,"threads_started":1}
I20260806 15:59:24.896171 30727 tablet_service.cc:1467] Processing CreateTablet for tablet dea3ba28a38b4abcab236e1a4658a50f (DEFAULT_TABLE table=test-table [id=a2b19d9c24ae4f8796d12713f262cb08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:24.896184 31001 tablet_service.cc:1467] Processing CreateTablet for tablet dea3ba28a38b4abcab236e1a4658a50f (DEFAULT_TABLE table=test-table [id=a2b19d9c24ae4f8796d12713f262cb08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:24.897243 31001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dea3ba28a38b4abcab236e1a4658a50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:24.897261 30727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dea3ba28a38b4abcab236e1a4658a50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:24.897320 30863 tablet_service.cc:1467] Processing CreateTablet for tablet dea3ba28a38b4abcab236e1a4658a50f (DEFAULT_TABLE table=test-table [id=a2b19d9c24ae4f8796d12713f262cb08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:24.898216 30863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dea3ba28a38b4abcab236e1a4658a50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:24.903268 31089 tablet_bootstrap.cc:492] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae: Bootstrap starting.
I20260806 15:59:24.905022 31091 tablet_bootstrap.cc:492] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a: Bootstrap starting.
I20260806 15:59:24.905166 31090 tablet_bootstrap.cc:492] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083: Bootstrap starting.
I20260806 15:59:24.906032 31089 tablet_bootstrap.cc:654] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:24.906929 31089 log.cc:824] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:24.907115 31090 tablet_bootstrap.cc:654] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:24.907135 31091 tablet_bootstrap.cc:654] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:24.907871 31091 log.cc:824] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:24.907924 31090 log.cc:824] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:24.909013 31089 tablet_bootstrap.cc:492] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae: No bootstrap required, opened a new log
I20260806 15:59:24.909217 31089 ts_tablet_manager.cc:1397] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 15:59:24.909730 31091 tablet_bootstrap.cc:492] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a: No bootstrap required, opened a new log
I20260806 15:59:24.909730 31090 tablet_bootstrap.cc:492] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083: No bootstrap required, opened a new log
I20260806 15:59:24.909902 31090 ts_tablet_manager.cc:1397] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 15:59:24.909902 31091 ts_tablet_manager.cc:1397] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 15:59:24.912854 31089 raft_consensus.cc:348] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66194b488877459d98a0363508d56fae" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34349 } } peers { permanent_uuid: "7d607ed3c17e4d1cb69eadad32643083" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 39467 } } peers { permanent_uuid: "fc73268a8a8a4dccbb947ac7b8f0507a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39961 } }
I20260806 15:59:24.913094 31089 raft_consensus.cc:374] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:24.913156 31089 raft_consensus.cc:729] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66194b488877459d98a0363508d56fae, State: Initialized, Role: FOLLOWER
I20260806 15:59:24.913589 31089 consensus_queue.cc:260] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66194b488877459d98a0363508d56fae" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34349 } } peers { permanent_uuid: "7d607ed3c17e4d1cb69eadad32643083" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 39467 } } peers { permanent_uuid: "fc73268a8a8a4dccbb947ac7b8f0507a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39961 } }
I20260806 15:59:24.914192 30795 heartbeater.cc:503] Master 127.25.31.190:37947 was elected leader, sending a full tablet report...
I20260806 15:59:24.914356 31090 raft_consensus.cc:348] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66194b488877459d98a0363508d56fae" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34349 } } peers { permanent_uuid: "7d607ed3c17e4d1cb69eadad32643083" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 39467 } } peers { permanent_uuid: "fc73268a8a8a4dccbb947ac7b8f0507a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39961 } }
I20260806 15:59:24.914356 31091 raft_consensus.cc:348] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66194b488877459d98a0363508d56fae" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34349 } } peers { permanent_uuid: "7d607ed3c17e4d1cb69eadad32643083" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 39467 } } peers { permanent_uuid: "fc73268a8a8a4dccbb947ac7b8f0507a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39961 } }
I20260806 15:59:24.914575 31089 ts_tablet_manager.cc:1428] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260806 15:59:24.914590 31090 raft_consensus.cc:374] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:24.914590 31091 raft_consensus.cc:374] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:24.914642 31091 raft_consensus.cc:729] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc73268a8a8a4dccbb947ac7b8f0507a, State: Initialized, Role: FOLLOWER
I20260806 15:59:24.914651 31090 raft_consensus.cc:729] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d607ed3c17e4d1cb69eadad32643083, State: Initialized, Role: FOLLOWER
I20260806 15:59:24.915045 31091 consensus_queue.cc:260] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66194b488877459d98a0363508d56fae" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34349 } } peers { permanent_uuid: "7d607ed3c17e4d1cb69eadad32643083" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 39467 } } peers { permanent_uuid: "fc73268a8a8a4dccbb947ac7b8f0507a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39961 } }
I20260806 15:59:24.915045 31090 consensus_queue.cc:260] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66194b488877459d98a0363508d56fae" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34349 } } peers { permanent_uuid: "7d607ed3c17e4d1cb69eadad32643083" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 39467 } } peers { permanent_uuid: "fc73268a8a8a4dccbb947ac7b8f0507a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39961 } }
I20260806 15:59:24.915977 31091 ts_tablet_manager.cc:1428] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 15:59:24.915988 31090 ts_tablet_manager.cc:1428] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260806 15:59:24.916975 31070 heartbeater.cc:503] Master 127.25.31.190:37947 was elected leader, sending a full tablet report...
I20260806 15:59:24.917840 30931 heartbeater.cc:503] Master 127.25.31.190:37947 was elected leader, sending a full tablet report...
W20260806 15:59:24.936019 30932 tablet.cc:2369] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:59:25.006477 30796 tablet.cc:2369] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:59:25.112469 31071 tablet.cc:2369] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:59:25.304203 31096 raft_consensus.cc:482] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:25.304478 31096 raft_consensus.cc:504] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66194b488877459d98a0363508d56fae" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34349 } } peers { permanent_uuid: "7d607ed3c17e4d1cb69eadad32643083" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 39467 } } peers { permanent_uuid: "fc73268a8a8a4dccbb947ac7b8f0507a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39961 } }
I20260806 15:59:25.305683 31096 leader_election.cc:288] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66194b488877459d98a0363508d56fae (127.25.31.129:34349), 7d607ed3c17e4d1cb69eadad32643083 (127.25.31.131:39467)
I20260806 15:59:25.308010 31085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:25.306089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:40747 (local address 127.25.31.131:39467)
0806 15:59:25.306233 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:25.306237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:25.306562 (+ 325us) server_negotiation.cc:407] Connection header received
0806 15:59:25.306614 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:25.306617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:25.306652 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:25.306742 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:25.307585 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:25.307588 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:25.307591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:25.307604 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:25.307627 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:25.307638 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:25.307729 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:25.307827 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 15:59:25.307870 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 15:59:25.308001 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:25.306017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:58785 (local address 127.25.31.129:34349)
0806 15:59:25.306151 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:25.306156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:25.306473 (+ 317us) server_negotiation.cc:407] Connection header received
0806 15:59:25.306579 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:25.306586 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:25.306647 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:25.306745 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:25.307585 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:25.307588 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:25.307591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:25.307605 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:25.307627 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:25.307637 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:25.307729 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:25.307831 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 15:59:25.307870 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 15:59:25.308048 31098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:25.305935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:34349 (local address 127.25.31.130:58785)
0806 15:59:25.306346 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:25.306362 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:25.306474 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:25.306783 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:25.306786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:25.306803 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:25.307382 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:59:25.307385 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:25.307400 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:59:25.307433 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:25.307730 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:25.307733 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:25.307736 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:59:25.307818 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 15:59:25.307883 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":19,"thread_start_us":111,"threads_started":1}
I20260806 15:59:25.308243 31099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:25.305953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:39467 (local address 127.25.31.130:40747)
0806 15:59:25.306434 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:25.306447 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:25.306539 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:25.306783 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:25.306786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:25.306803 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:25.307416 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:59:25.307417 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:25.307425 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:59:25.307441 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:25.307730 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:25.307733 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:25.307736 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:59:25.307818 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 15:59:25.307872 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":96,"threads_started":1}
I20260806 15:59:25.308641 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea3ba28a38b4abcab236e1a4658a50f" candidate_uuid: "fc73268a8a8a4dccbb947ac7b8f0507a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66194b488877459d98a0363508d56fae" is_pre_election: true
I20260806 15:59:25.308941 30747 raft_consensus.cc:2454] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc73268a8a8a4dccbb947ac7b8f0507a in term 0.
I20260806 15:59:25.308921 31021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea3ba28a38b4abcab236e1a4658a50f" candidate_uuid: "fc73268a8a8a4dccbb947ac7b8f0507a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d607ed3c17e4d1cb69eadad32643083" is_pre_election: true
I20260806 15:59:25.309203 31021 raft_consensus.cc:2454] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc73268a8a8a4dccbb947ac7b8f0507a in term 0.
I20260806 15:59:25.309465 30818 leader_election.cc:302] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66194b488877459d98a0363508d56fae, fc73268a8a8a4dccbb947ac7b8f0507a; no voters:
I20260806 15:59:25.309747 31096 raft_consensus.cc:2781] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:25.309832 31096 raft_consensus.cc:482] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:25.309875 31096 raft_consensus.cc:3037] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:25.310997 31096 raft_consensus.cc:504] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66194b488877459d98a0363508d56fae" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34349 } } peers { permanent_uuid: "7d607ed3c17e4d1cb69eadad32643083" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 39467 } } peers { permanent_uuid: "fc73268a8a8a4dccbb947ac7b8f0507a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39961 } }
I20260806 15:59:25.311335 31096 leader_election.cc:288] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [CANDIDATE]: Term 1 election: Requested vote from peers 66194b488877459d98a0363508d56fae (127.25.31.129:34349), 7d607ed3c17e4d1cb69eadad32643083 (127.25.31.131:39467)
I20260806 15:59:25.311684 30747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea3ba28a38b4abcab236e1a4658a50f" candidate_uuid: "fc73268a8a8a4dccbb947ac7b8f0507a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66194b488877459d98a0363508d56fae"
I20260806 15:59:25.311738 31021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dea3ba28a38b4abcab236e1a4658a50f" candidate_uuid: "fc73268a8a8a4dccbb947ac7b8f0507a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d607ed3c17e4d1cb69eadad32643083"
I20260806 15:59:25.311805 30747 raft_consensus.cc:3037] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:25.311944 31021 raft_consensus.cc:3037] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:25.312860 30747 raft_consensus.cc:2454] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc73268a8a8a4dccbb947ac7b8f0507a in term 1.
I20260806 15:59:25.313055 31021 raft_consensus.cc:2454] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc73268a8a8a4dccbb947ac7b8f0507a in term 1.
I20260806 15:59:25.313169 30818 leader_election.cc:302] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66194b488877459d98a0363508d56fae, fc73268a8a8a4dccbb947ac7b8f0507a; no voters:
I20260806 15:59:25.313337 31096 raft_consensus.cc:2781] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:25.313560 31096 raft_consensus.cc:686] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [term 1 LEADER]: Becoming Leader. State: Replica: fc73268a8a8a4dccbb947ac7b8f0507a, State: Running, Role: LEADER
I20260806 15:59:25.313862 31096 consensus_queue.cc:237] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66194b488877459d98a0363508d56fae" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34349 } } peers { permanent_uuid: "7d607ed3c17e4d1cb69eadad32643083" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 39467 } } peers { permanent_uuid: "fc73268a8a8a4dccbb947ac7b8f0507a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39961 } }
I20260806 15:59:25.317016 30597 catalog_manager.cc:5495] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a reported cstate change: term changed from 0 to 1, leader changed from <none> to fc73268a8a8a4dccbb947ac7b8f0507a (127.25.31.130). New cstate: current_term: 1 leader_uuid: "fc73268a8a8a4dccbb947ac7b8f0507a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66194b488877459d98a0363508d56fae" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d607ed3c17e4d1cb69eadad32643083" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 39467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc73268a8a8a4dccbb947ac7b8f0507a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39961 } health_report { overall_health: HEALTHY } } }
I20260806 15:59:25.344462 31087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:25.342956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36376 (local address 127.25.31.130:39961)
0806 15:59:25.343110 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:25.343113 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:25.343149 (+ 36us) server_negotiation.cc:407] Connection header received
0806 15:59:25.343221 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:25.343223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:25.343281 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:25.343356 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:25.344120 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:25.344122 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:25.344124 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:25.344134 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:25.344153 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:25.344160 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:25.344231 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:25.344313 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 15:59:25.344349 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 15:59:25.350366 31021 raft_consensus.cc:1261] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083 [term 1 FOLLOWER]: Refusing update from remote peer fc73268a8a8a4dccbb947ac7b8f0507a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:25.350386 30747 raft_consensus.cc:1261] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae [term 1 FOLLOWER]: Refusing update from remote peer fc73268a8a8a4dccbb947ac7b8f0507a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:25.351105 31096 consensus_queue.cc:1035] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66194b488877459d98a0363508d56fae" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:25.351323 31100 consensus_queue.cc:1035] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d607ed3c17e4d1cb69eadad32643083" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 39467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:25.358862 31105 mvcc.cc:205] Tried to move back new op lower bound from 7315586930067365888 to 7315586929928212480. Current Snapshot: MvccSnapshot[applied={T|T < 7315586930067365888}]
I20260806 15:59:25.360172 31104 mvcc.cc:205] Tried to move back new op lower bound from 7315586930067365888 to 7315586929928212480. Current Snapshot: MvccSnapshot[applied={T|T < 7315586930067365888}]
I20260806 15:59:25.366863 30597 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:25.366998 30597 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:25.369385 30597 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2a917895426459aa6aaa9fdb795898d: Sending DeleteTablet for 3 replicas of tablet dea3ba28a38b4abcab236e1a4658a50f
I20260806 15:59:25.370116 30727 tablet_service.cc:1514] Processing DeleteTablet for tablet dea3ba28a38b4abcab236e1a4658a50f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:25 UTC) from {username='slave'} at 127.0.0.1:40636
I20260806 15:59:25.370236 30863 tablet_service.cc:1514] Processing DeleteTablet for tablet dea3ba28a38b4abcab236e1a4658a50f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:25 UTC) from {username='slave'} at 127.0.0.1:36372
I20260806 15:59:25.370231 31001 tablet_service.cc:1514] Processing DeleteTablet for tablet dea3ba28a38b4abcab236e1a4658a50f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:25 UTC) from {username='slave'} at 127.0.0.1:50844
I20260806 15:59:25.370636 31114 tablet_replica.cc:330] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae: stopping tablet replica
I20260806 15:59:25.370954 31114 raft_consensus.cc:2229] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:25.371259 31114 raft_consensus.cc:2258] T dea3ba28a38b4abcab236e1a4658a50f P 66194b488877459d98a0363508d56fae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:25.371308 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 30660
I20260806 15:59:25.373811 31115 tablet_replica.cc:330] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a: stopping tablet replica
I20260806 15:59:25.374069 31115 raft_consensus.cc:2229] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:59:25.374473 31115 raft_consensus.cc:2258] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:25.375699 31115 ts_tablet_manager.cc:1905] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:25.378082 31116 tablet_replica.cc:330] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083: stopping tablet replica
I20260806 15:59:25.378190 31115 ts_tablet_manager.cc:1918] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:25.378353 31115 log.cc:1196] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/dea3ba28a38b4abcab236e1a4658a50f
I20260806 15:59:25.378443 31116 raft_consensus.cc:2229] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:25.378755 31116 raft_consensus.cc:2258] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:25.378886 31115 ts_tablet_manager.cc:1939] T dea3ba28a38b4abcab236e1a4658a50f P fc73268a8a8a4dccbb947ac7b8f0507a: Deleting consensus metadata
I20260806 15:59:25.380110 30583 catalog_manager.cc:4841] TS fc73268a8a8a4dccbb947ac7b8f0507a (127.25.31.130:39961): tablet dea3ba28a38b4abcab236e1a4658a50f (table test-table [id=a2b19d9c24ae4f8796d12713f262cb08]) successfully deleted
I20260806 15:59:25.380182 31116 ts_tablet_manager.cc:1905] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 15:59:25.380112 30584 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:34349 (error 108)
I20260806 15:59:25.380491 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 30798
I20260806 15:59:25.382800 31086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:25.382454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:34349 (local address 127.0.0.1:40652)
0806 15:59:25.382643 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:25.382718 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:34349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260806 15:59:25.382970 30584 catalog_manager.cc:4584] TS 66194b488877459d98a0363508d56fae (127.25.31.129:34349): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dea3ba28a38b4abcab236e1a4658a50f: Network error: Client connection negotiation failed: client connection to 127.25.31.129:34349: connect: Connection refused (error 111)
I20260806 15:59:25.383519 31116 ts_tablet_manager.cc:1918] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:25.383597 31116 log.cc:1196] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/dea3ba28a38b4abcab236e1a4658a50f
I20260806 15:59:25.383888 31116 ts_tablet_manager.cc:1939] T dea3ba28a38b4abcab236e1a4658a50f P 7d607ed3c17e4d1cb69eadad32643083: Deleting consensus metadata
I20260806 15:59:25.384894 30583 catalog_manager.cc:4841] TS 7d607ed3c17e4d1cb69eadad32643083 (127.25.31.131:39467): tablet dea3ba28a38b4abcab236e1a4658a50f (table test-table [id=a2b19d9c24ae4f8796d12713f262cb08]) successfully deleted
I20260806 15:59:25.387676 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 30934
I20260806 15:59:25.393424 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 30565
2026-08-06T15:59:25Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1317 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 15:59:25.415416 25726 mini_oidc.cc:152] Starting JWKS server
I20260806 15:59:25.417018 25726 webserver.cc:460] Webserver started at https://127.0.0.1:43129/ using document root <none> and password file <none>
I20260806 15:59:25.417153 25726 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 15:59:25.417415 25726 webserver.cc:460] Webserver started at http://127.0.0.1:40399/ using document root <none> and password file <none>
2026-08-06T15:59:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:25Z Disabled control of system clock
I20260806 15:59:25.426493 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:43717
--webserver_interface=127.25.31.190
--webserver_port=0
--builtin_ntp_servers=127.25.31.148:41887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:43717
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43129/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/testchainca.cert with env {}
W20260806 15:59:25.535101 31124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:25.535354 31124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:25.535436 31124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:25.539430 31124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:25.539484 31124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:25.539500 31124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:25.539516 31124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:25.539556 31124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 15:59:25.539593 31124 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 15:59:25.539619 31124 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43129/jwks.json
I20260806 15:59:25.544872 31124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:41887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:43717
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:43717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43129/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:25.545938 31124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:25.547081 31124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:25.555452 31129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:25.555466 31132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:25.555428 31130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:25.555776 31124 server_base.cc:1034] running on GCE node
I20260806 15:59:25.556344 31124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:25.557407 31124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:25.558604 31124 hybrid_clock.cc:648] HybridClock initialized: now 1786031965558573 us; error 42 us; skew 500 ppm
I20260806 15:59:25.560436 31124 webserver.cc:460] Webserver started at http://127.25.31.190:45279/ using document root <none> and password file <none>
I20260806 15:59:25.561008 31124 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:25.561103 31124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:25.561331 31124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:25.563383 31124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "c6689634031a4111975b428b85a4ff0b"
format_stamp: "Formatted at 2026-08-06 15:59:25 on dist-test-slave-tm1g"
server_key: "aadb2c10360b38d9e503509dfe1fb520"
server_key_iv: "db4554f9bda0afefdbf93b352916bdf2"
server_key_version: "encryptionkey@0"
I20260806 15:59:25.563966 31124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "c6689634031a4111975b428b85a4ff0b"
format_stamp: "Formatted at 2026-08-06 15:59:25 on dist-test-slave-tm1g"
server_key: "aadb2c10360b38d9e503509dfe1fb520"
server_key_iv: "db4554f9bda0afefdbf93b352916bdf2"
server_key_version: "encryptionkey@0"
I20260806 15:59:25.567540 31124 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 15:59:25.569808 31138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:25.571065 31124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:25.571208 31124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "c6689634031a4111975b428b85a4ff0b"
format_stamp: "Formatted at 2026-08-06 15:59:25 on dist-test-slave-tm1g"
server_key: "aadb2c10360b38d9e503509dfe1fb520"
server_key_iv: "db4554f9bda0afefdbf93b352916bdf2"
server_key_version: "encryptionkey@0"
I20260806 15:59:25.571316 31124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:25.584295 31124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:25.584900 31124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:25.585095 31124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:25.591945 31124 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:43717
I20260806 15:59:25.591954 31192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:43717 every 8 connection(s)
I20260806 15:59:25.592841 31124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:25.596026 31193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:25.601337 31193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b: Bootstrap starting.
I20260806 15:59:25.601826 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 31124
I20260806 15:59:25.601948 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:25.602262 25726 external_mini_cluster.cc:1428] Setting key 80f1063a1c2112f3cf297ab7d4359f0a
I20260806 15:59:25.604059 31193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:25.604779 31193 log.cc:824] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:25.606876 31193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b: No bootstrap required, opened a new log
I20260806 15:59:25.611094 31193 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6689634031a4111975b428b85a4ff0b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43717 } }
I20260806 15:59:25.611295 31193 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:25.611356 31193 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6689634031a4111975b428b85a4ff0b, State: Initialized, Role: FOLLOWER
I20260806 15:59:25.611644 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:25.603530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52462 (local address 127.25.31.190:43717)
0806 15:59:25.604223 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:25.604242 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:25.604278 (+ 36us) server_negotiation.cc:407] Connection header received
0806 15:59:25.604897 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:25.604927 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:25.605411 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:25.605729 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:25.606058 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:25.607119 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:25.608317 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:25.608766 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:25.608877 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:25.609727 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:25.609732 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:25.609743 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:25.610114 (+ 371us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:25.610168 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:25.610187 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:25.610261 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:25.610509 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 15:59:25.610812 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":165,"threads_started":1}
I20260806 15:59:25.611799 31193 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6689634031a4111975b428b85a4ff0b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43717 } }
I20260806 15:59:25.611979 31193 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:25.612059 31193 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:25.612185 31193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:25.613170 31193 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6689634031a4111975b428b85a4ff0b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43717 } }
I20260806 15:59:25.613513 31193 leader_election.cc:302] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6689634031a4111975b428b85a4ff0b; no voters:
I20260806 15:59:25.613870 31193 leader_election.cc:288] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:25.613981 31199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:25.614204 31199 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [term 1 LEADER]: Becoming Leader. State: Replica: c6689634031a4111975b428b85a4ff0b, State: Running, Role: LEADER
I20260806 15:59:25.614611 31199 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6689634031a4111975b428b85a4ff0b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43717 } }
I20260806 15:59:25.614748 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:59:25.615140 31193 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:25.615736 31199 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6689634031a4111975b428b85a4ff0b. Latest consensus state: current_term: 1 leader_uuid: "c6689634031a4111975b428b85a4ff0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6689634031a4111975b428b85a4ff0b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43717 } } }
I20260806 15:59:25.615911 31199 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:25.616196 31201 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6689634031a4111975b428b85a4ff0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6689634031a4111975b428b85a4ff0b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43717 } } }
I20260806 15:59:25.616278 31201 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:25.616719 31204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:25.620210 31204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:25.625741 31204 catalog_manager.cc:1347] Generated new cluster ID: f57fe80670a5496f8f94f9fdf8f6930e
I20260806 15:59:25.625818 31204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:25.642360 31204 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:25.643362 31204 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:25.649362 31204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6689634031a4111975b428b85a4ff0b: Generated new TSK 0
I20260806 15:59:25.649950 31204 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 15:59:25.682045 31226 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 15:59:25.682150 31226 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 15:59:25.682710 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:25.658266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52476 (local address 127.25.31.190:43717)
0806 15:59:25.658415 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:25.658419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:25.658672 (+ 253us) server_negotiation.cc:407] Connection header received
0806 15:59:25.658874 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:25.658876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:25.658919 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:25.658998 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0806 15:59:25.659299 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:25.659835 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:25.661045 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:25.661256 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:25.661325 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:25.661697 (+ 372us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 15:59:25.682380 (+ 20683us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43129/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 15:59:25.682602 (+ 222us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52476: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43129/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 15:59:25.682814 31197 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52476: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43129/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 15:59:25.684288 31223 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:25.658197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:43717 (local address 127.0.0.1:52476)
0806 15:59:25.658576 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:25.658592 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:25.658729 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:25.659046 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:25.659049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:25.659057 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 15:59:25.659194 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:25.659201 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:25.659946 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:25.659949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:25.660925 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:25.660935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:25.661373 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:25.661377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:25.661479 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:25.661533 (+ 54us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 15:59:25.682594 (+ 21061us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43129/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 15:59:25.682836 (+ 242us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.31.190:43717: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43129/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":137,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.25.31.190:43717: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43129/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 15:59:25.685110 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 31124
2026-08-06T15:59:25Z chronyd exiting
I20260806 15:59:25.723407 25726 test_util.cc:175] -----------------------------------------------
I20260806 15:59:25.723498 25726 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786031939584994-25726-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (309 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 15:59:25.724651 25726 mini_oidc.cc:152] Starting JWKS server
I20260806 15:59:25.725543 25726 webserver.cc:460] Webserver started at https://127.0.0.1:36563/ using document root <none> and password file <none>
I20260806 15:59:25.725651 25726 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 15:59:25.725926 25726 webserver.cc:460] Webserver started at http://127.0.0.1:43243/ using document root <none> and password file <none>
2026-08-06T15:59:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:25Z Disabled control of system clock
I20260806 15:59:25.734814 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42987
--webserver_interface=127.25.31.190
--webserver_port=0
--builtin_ntp_servers=127.25.31.148:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:42987
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36563/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/testchainca.cert with env {}
W20260806 15:59:25.846447 31232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:25.846730 31232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:25.846810 31232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:25.850941 31232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:25.851017 31232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:25.851047 31232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:25.851104 31232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:25.851147 31232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 15:59:25.851182 31232 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 15:59:25.851209 31232 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36563/jwks.json
I20260806 15:59:25.856431 31232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:42987
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36563/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:25.857427 31232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:25.858489 31232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:25.866849 31237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:25.866861 31238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:25.866976 31232 server_base.cc:1034] running on GCE node
W20260806 15:59:25.866832 31240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:25.867719 31232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:25.868775 31232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:25.869995 31232 hybrid_clock.cc:648] HybridClock initialized: now 1786031965869979 us; error 33 us; skew 500 ppm
I20260806 15:59:25.871826 31232 webserver.cc:460] Webserver started at http://127.25.31.190:38747/ using document root <none> and password file <none>
I20260806 15:59:25.872386 31232 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:25.872479 31232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:25.872709 31232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:25.874738 31232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "b14147220f984177b118c6abc13b59e8"
format_stamp: "Formatted at 2026-08-06 15:59:25 on dist-test-slave-tm1g"
server_key: "0ec1eebb724b0822f66bc12cc278458f"
server_key_iv: "539c0fe9b59ce43dd8e8cc8155ff4f4a"
server_key_version: "encryptionkey@0"
I20260806 15:59:25.875301 31232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "b14147220f984177b118c6abc13b59e8"
format_stamp: "Formatted at 2026-08-06 15:59:25 on dist-test-slave-tm1g"
server_key: "0ec1eebb724b0822f66bc12cc278458f"
server_key_iv: "539c0fe9b59ce43dd8e8cc8155ff4f4a"
server_key_version: "encryptionkey@0"
I20260806 15:59:25.878819 31232 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 15:59:25.881162 31246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:25.882404 31232 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:25.882531 31232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "b14147220f984177b118c6abc13b59e8"
format_stamp: "Formatted at 2026-08-06 15:59:25 on dist-test-slave-tm1g"
server_key: "0ec1eebb724b0822f66bc12cc278458f"
server_key_iv: "539c0fe9b59ce43dd8e8cc8155ff4f4a"
server_key_version: "encryptionkey@0"
I20260806 15:59:25.882642 31232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:25.898789 31232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:25.899482 31232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:25.899684 31232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:25.906641 31232 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:42987
I20260806 15:59:25.906664 31299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:42987 every 8 connection(s)
I20260806 15:59:25.907552 31232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:25.910235 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 31232
I20260806 15:59:25.910377 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:25.910692 25726 external_mini_cluster.cc:1428] Setting key 24ebc49158612208dc41eb06e8526fa5
I20260806 15:59:25.910934 31300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:25.918157 31300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8: Bootstrap starting.
I20260806 15:59:25.920747 31300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:25.921427 31303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:25.912054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35232 (local address 127.25.31.190:42987)
0806 15:59:25.912726 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:25.912746 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:25.912789 (+ 43us) server_negotiation.cc:407] Connection header received
0806 15:59:25.913701 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:25.913727 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:25.914229 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:25.914622 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:25.915076 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:25.916486 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:25.917409 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:25.918080 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:25.918244 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:25.919302 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:25.919308 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:25.919323 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:25.919799 (+ 476us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:25.919878 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:25.919904 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:25.920002 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:25.920177 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 15:59:25.920437 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":120,"threads_started":1}
I20260806 15:59:25.921635 31300 log.cc:824] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:25.923635 31300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8: No bootstrap required, opened a new log
I20260806 15:59:25.924165 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:59:25.927386 31300 raft_consensus.cc:348] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14147220f984177b118c6abc13b59e8" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42987 } }
I20260806 15:59:25.927589 31300 raft_consensus.cc:374] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:25.927665 31300 raft_consensus.cc:729] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b14147220f984177b118c6abc13b59e8, State: Initialized, Role: FOLLOWER
I20260806 15:59:25.928117 31300 consensus_queue.cc:260] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14147220f984177b118c6abc13b59e8" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42987 } }
I20260806 15:59:25.928262 31300 raft_consensus.cc:388] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:25.928356 31300 raft_consensus.cc:482] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:25.928474 31300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:25.929291 31300 raft_consensus.cc:504] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14147220f984177b118c6abc13b59e8" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42987 } }
I20260806 15:59:25.929623 31300 leader_election.cc:302] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b14147220f984177b118c6abc13b59e8; no voters:
I20260806 15:59:25.929919 31300 leader_election.cc:288] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:25.930081 31305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:25.930367 31305 raft_consensus.cc:686] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [term 1 LEADER]: Becoming Leader. State: Replica: b14147220f984177b118c6abc13b59e8, State: Running, Role: LEADER
I20260806 15:59:25.930683 31305 consensus_queue.cc:237] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14147220f984177b118c6abc13b59e8" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42987 } }
I20260806 15:59:25.931118 31300 sys_catalog.cc:564] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:25.932344 31307 sys_catalog.cc:455] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b14147220f984177b118c6abc13b59e8. Latest consensus state: current_term: 1 leader_uuid: "b14147220f984177b118c6abc13b59e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14147220f984177b118c6abc13b59e8" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42987 } } }
I20260806 15:59:25.932361 31306 sys_catalog.cc:455] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b14147220f984177b118c6abc13b59e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14147220f984177b118c6abc13b59e8" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42987 } } }
I20260806 15:59:25.932502 31307 sys_catalog.cc:458] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:25.932539 31306 sys_catalog.cc:458] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:25.932951 31314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:25.935673 31314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:25.940717 31314 catalog_manager.cc:1347] Generated new cluster ID: d634119a6ebd49629880a5f2fc3a8d8a
I20260806 15:59:25.940781 31314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:25.956996 31314 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:25.958171 31314 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:25.962670 31314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b14147220f984177b118c6abc13b59e8: Generated new TSK 0
I20260806 15:59:25.963385 31314 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 15:59:25.990250 31330 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 15:59:25.990365 31330 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 15:59:25.990981 31303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:25.974676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35236 (local address 127.25.31.190:42987)
0806 15:59:25.974815 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:25.974820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:25.975035 (+ 215us) server_negotiation.cc:407] Connection header received
0806 15:59:25.975183 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:25.975186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:25.975230 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:25.975327 (+ 97us) server_negotiation.cc:226] Negotiated authn=JWT
0806 15:59:25.975565 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:25.976084 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:25.977328 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:25.977535 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:25.977614 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:25.977966 (+ 352us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 15:59:25.990658 (+ 12692us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36563/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 15:59:25.990869 (+ 211us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35236: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36563/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 15:59:25.991065 31303 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35236: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36563/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 15:59:25.991232 31329 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:25.974617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42987 (local address 127.0.0.1:35236)
0806 15:59:25.974933 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:25.974948 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:25.975069 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:25.975335 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:25.975340 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:25.975349 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 15:59:25.975470 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:25.975476 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:25.976189 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:25.976193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:25.977183 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:25.977194 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:25.977646 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:25.977650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:25.977749 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:25.977776 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 15:59:25.990845 (+ 13069us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36563/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 15:59:25.991048 (+ 203us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.31.190:42987: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36563/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":134,"threads_started":1}
I20260806 15:59:25.991708 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 31232
2026-08-06T15:59:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (307 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 15:59:26.031919 25726 mini_oidc.cc:152] Starting JWKS server
I20260806 15:59:26.032804 25726 webserver.cc:460] Webserver started at https://127.0.0.1:46721/ using document root <none> and password file <none>
I20260806 15:59:26.032936 25726 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 15:59:26.033205 25726 webserver.cc:460] Webserver started at http://127.0.0.1:46031/ using document root <none> and password file <none>
2026-08-06T15:59:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:26Z Disabled control of system clock
I20260806 15:59:26.042171 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:43595
--webserver_interface=127.25.31.190
--webserver_port=0
--builtin_ntp_servers=127.25.31.148:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:43595
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46721/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 15:59:26.150560 31336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:26.150802 31336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:26.150840 31336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:26.154892 31336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:26.154942 31336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:26.154956 31336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:26.154973 31336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:26.154987 31336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 15:59:26.155002 31336 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 15:59:26.155015 31336 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46721/jwks.json
I20260806 15:59:26.159513 31336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:43595
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:43595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46721/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:26.160414 31336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:26.161669 31336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:26.169850 31343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:26.169850 31345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:26.169916 31342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:26.169999 31336 server_base.cc:1034] running on GCE node
I20260806 15:59:26.170694 31336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:26.171697 31336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:26.172858 31336 hybrid_clock.cc:648] HybridClock initialized: now 1786031966172845 us; error 25 us; skew 500 ppm
I20260806 15:59:26.174767 31336 webserver.cc:460] Webserver started at http://127.25.31.190:46211/ using document root <none> and password file <none>
I20260806 15:59:26.175310 31336 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:26.175371 31336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:26.175650 31336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:26.177587 31336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "6a39491b513d46678fedbf4899e6a133"
format_stamp: "Formatted at 2026-08-06 15:59:26 on dist-test-slave-tm1g"
server_key: "2dde9cebbf464ad730c0851a715786bd"
server_key_iv: "0d3d82f77d922fca59de2f810690d8e7"
server_key_version: "encryptionkey@0"
I20260806 15:59:26.178139 31336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "6a39491b513d46678fedbf4899e6a133"
format_stamp: "Formatted at 2026-08-06 15:59:26 on dist-test-slave-tm1g"
server_key: "2dde9cebbf464ad730c0851a715786bd"
server_key_iv: "0d3d82f77d922fca59de2f810690d8e7"
server_key_version: "encryptionkey@0"
I20260806 15:59:26.181586 31336 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 15:59:26.183916 31351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:26.185015 31336 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 15:59:26.185143 31336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "6a39491b513d46678fedbf4899e6a133"
format_stamp: "Formatted at 2026-08-06 15:59:26 on dist-test-slave-tm1g"
server_key: "2dde9cebbf464ad730c0851a715786bd"
server_key_iv: "0d3d82f77d922fca59de2f810690d8e7"
server_key_version: "encryptionkey@0"
I20260806 15:59:26.185241 31336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:26.199767 31336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:26.200385 31336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:26.200577 31336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:26.207353 31407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:43595 every 8 connection(s)
I20260806 15:59:26.207352 31336 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:43595
I20260806 15:59:26.208348 31336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:26.211334 31409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:26.216526 31409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133: Bootstrap starting.
I20260806 15:59:26.217458 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 31336
I20260806 15:59:26.217563 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:26.217885 25726 external_mini_cluster.cc:1428] Setting key 07f4b6c1956c60fd1aeaaf305b7dac97
I20260806 15:59:26.219200 31409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:26.219923 31409 log.cc:824] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:26.221869 31409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133: No bootstrap required, opened a new log
I20260806 15:59:26.226050 31409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a39491b513d46678fedbf4899e6a133" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43595 } }
I20260806 15:59:26.226271 31409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:26.226374 31409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a39491b513d46678fedbf4899e6a133, State: Initialized, Role: FOLLOWER
I20260806 15:59:26.226563 31412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:26.218967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57038 (local address 127.25.31.190:43595)
0806 15:59:26.219540 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:26.219554 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:26.219587 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:59:26.220237 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:26.220255 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:26.220568 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:26.220833 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:26.221160 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:26.222146 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:26.223354 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:26.223800 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:26.223916 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:26.224760 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:26.224765 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:26.224777 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:26.225149 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:26.225201 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:26.225220 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:26.225295 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:26.225481 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 15:59:26.225757 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":119,"threads_started":1}
I20260806 15:59:26.226869 31409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a39491b513d46678fedbf4899e6a133" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43595 } }
I20260806 15:59:26.227034 31409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:26.227101 31409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:26.227195 31409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:26.228084 31409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a39491b513d46678fedbf4899e6a133" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43595 } }
I20260806 15:59:26.228430 31409 leader_election.cc:302] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a39491b513d46678fedbf4899e6a133; no voters:
I20260806 15:59:26.228806 31409 leader_election.cc:288] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:26.228912 31414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:26.229111 31414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [term 1 LEADER]: Becoming Leader. State: Replica: 6a39491b513d46678fedbf4899e6a133, State: Running, Role: LEADER
I20260806 15:59:26.229331 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:59:26.229466 31414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a39491b513d46678fedbf4899e6a133" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43595 } }
I20260806 15:59:26.229902 31409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:26.230722 31414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a39491b513d46678fedbf4899e6a133. Latest consensus state: current_term: 1 leader_uuid: "6a39491b513d46678fedbf4899e6a133" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a39491b513d46678fedbf4899e6a133" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43595 } } }
I20260806 15:59:26.231438 31414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:26.231325 31415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a39491b513d46678fedbf4899e6a133" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a39491b513d46678fedbf4899e6a133" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43595 } } }
I20260806 15:59:26.231771 31415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133 [sys.catalog]: This master's current role is: LEADER
W20260806 15:59:26.234618 31429 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:59:26.234681 31429 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:59:26.234755 31427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:26.235632 31427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:26.241115 31427 catalog_manager.cc:1347] Generated new cluster ID: 476295c2530a4ae5875e76bd859070cd
I20260806 15:59:26.241179 31427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:26.254954 31427 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:26.256101 31427 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:26.268132 31427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a39491b513d46678fedbf4899e6a133: Generated new TSK 0
I20260806 15:59:26.268889 31427 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:59:26.286434 31412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:26.281082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57040 (local address 127.25.31.190:43595)
0806 15:59:26.281227 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:26.281232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:26.281386 (+ 154us) server_negotiation.cc:407] Connection header received
0806 15:59:26.281534 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:26.281537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:26.281595 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:26.281715 (+ 120us) server_negotiation.cc:226] Negotiated authn=JWT
0806 15:59:26.281996 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:26.282870 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:26.283753 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:26.284108 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:26.284209 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:26.284504 (+ 295us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 15:59:26.286013 (+ 1509us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46721/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 15:59:26.286241 (+ 228us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57040: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46721/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 15:59:26.286514 31412 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57040: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46721/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 15:59:26.288245 31438 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:26.280947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:43595 (local address 127.0.0.1:57040)
0806 15:59:26.281292 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:26.281305 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:26.281408 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:26.281744 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:26.281747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:26.281756 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 15:59:26.281870 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:26.281875 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:26.282992 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:26.282995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:26.283634 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:26.283642 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:26.284225 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:26.284228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:26.284305 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:26.284328 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 15:59:26.286189 (+ 1861us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46721/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 15:59:26.288063 (+ 1874us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.31.190:43595: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46721/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":1674,"thread_start_us":116,"threads_started":1}
I20260806 15:59:26.288827 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 31336
E20260806 15:59:26.300046 31439 webserver.cc:533] Webserver: sslize failed:
E20260806 15:59:26.300140 31439 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T15:59:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (298 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T15:59:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:26Z Disabled control of system clock
I20260806 15:59:26.338398 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:34669
--webserver_interface=127.25.31.190
--webserver_port=0
--builtin_ntp_servers=127.25.31.148:42765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:34669
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/testchainca.cert with env {}
W20260806 15:59:26.445356 31443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:26.445595 31443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:26.445632 31443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:26.449534 31443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:26.449589 31443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:26.449604 31443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:26.449623 31443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:26.449638 31443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 15:59:26.449694 31443 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 15:59:26.449726 31443 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 15:59:26.455087 31443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:42765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:34669
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:34669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:26.456037 31443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:26.457000 31443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:26.465322 31451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:26.465346 31448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:26.465323 31449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:26.465700 31443 server_base.cc:1034] running on GCE node
I20260806 15:59:26.466245 31443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:26.467252 31443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:26.468433 31443 hybrid_clock.cc:648] HybridClock initialized: now 1786031966468417 us; error 26 us; skew 500 ppm
I20260806 15:59:26.470127 31443 webserver.cc:460] Webserver started at http://127.25.31.190:46603/ using document root <none> and password file <none>
I20260806 15:59:26.470728 31443 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:26.470818 31443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:26.471050 31443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:26.473021 31443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "aeccca01dfa54605b832ff08fdbad525"
format_stamp: "Formatted at 2026-08-06 15:59:26 on dist-test-slave-tm1g"
server_key: "7b640a3a4f77ccaa922c303e043c3dbe"
server_key_iv: "fa72f30cd8a155208e8faf132e99f210"
server_key_version: "encryptionkey@0"
I20260806 15:59:26.473572 31443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "aeccca01dfa54605b832ff08fdbad525"
format_stamp: "Formatted at 2026-08-06 15:59:26 on dist-test-slave-tm1g"
server_key: "7b640a3a4f77ccaa922c303e043c3dbe"
server_key_iv: "fa72f30cd8a155208e8faf132e99f210"
server_key_version: "encryptionkey@0"
I20260806 15:59:26.476989 31443 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 15:59:26.480120 31457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:26.481691 31443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 15:59:26.481830 31443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "aeccca01dfa54605b832ff08fdbad525"
format_stamp: "Formatted at 2026-08-06 15:59:26 on dist-test-slave-tm1g"
server_key: "7b640a3a4f77ccaa922c303e043c3dbe"
server_key_iv: "fa72f30cd8a155208e8faf132e99f210"
server_key_version: "encryptionkey@0"
I20260806 15:59:26.481942 31443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:26.504876 31443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:26.505532 31443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:26.505715 31443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:26.513196 31443 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:34669
I20260806 15:59:26.513211 31513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:34669 every 8 connection(s)
I20260806 15:59:26.514161 31443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:26.516948 31514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:26.522236 31514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525: Bootstrap starting.
I20260806 15:59:26.523733 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 31443
I20260806 15:59:26.523830 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:26.524149 25726 external_mini_cluster.cc:1428] Setting key 514e2010655de680b8061a142e161794
I20260806 15:59:26.524816 31514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:26.525681 31514 log.cc:824] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:26.527916 31514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525: No bootstrap required, opened a new log
I20260806 15:59:26.532122 31514 raft_consensus.cc:348] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeccca01dfa54605b832ff08fdbad525" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34669 } }
I20260806 15:59:26.532414 31514 raft_consensus.cc:374] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:26.532476 31514 raft_consensus.cc:729] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeccca01dfa54605b832ff08fdbad525, State: Initialized, Role: FOLLOWER
I20260806 15:59:26.532950 31514 consensus_queue.cc:260] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeccca01dfa54605b832ff08fdbad525" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34669 } }
I20260806 15:59:26.533141 31514 raft_consensus.cc:388] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:26.533209 31514 raft_consensus.cc:482] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:26.533293 31514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:26.534317 31517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:26.525246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57668 (local address 127.25.31.190:34669)
0806 15:59:26.525967 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:26.525993 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:26.526040 (+ 47us) server_negotiation.cc:407] Connection header received
0806 15:59:26.526850 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:26.526873 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:26.527280 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:26.527630 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:26.528072 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:26.529572 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:26.530490 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:26.531296 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:26.531487 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:26.532247 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:26.532253 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:26.532267 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:26.532757 (+ 490us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:26.532826 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:26.532850 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:26.532953 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:26.533087 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 15:59:26.533359 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":142,"threads_started":1}
I20260806 15:59:26.534380 31514 raft_consensus.cc:504] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeccca01dfa54605b832ff08fdbad525" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34669 } }
I20260806 15:59:26.534713 31514 leader_election.cc:302] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aeccca01dfa54605b832ff08fdbad525; no voters:
I20260806 15:59:26.535018 31514 leader_election.cc:288] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:26.535125 31519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:26.535355 31519 raft_consensus.cc:686] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [term 1 LEADER]: Becoming Leader. State: Replica: aeccca01dfa54605b832ff08fdbad525, State: Running, Role: LEADER
I20260806 15:59:26.535645 31519 consensus_queue.cc:237] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeccca01dfa54605b832ff08fdbad525" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34669 } }
I20260806 15:59:26.536106 31514 sys_catalog.cc:564] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:26.536911 31521 sys_catalog.cc:455] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aeccca01dfa54605b832ff08fdbad525. Latest consensus state: current_term: 1 leader_uuid: "aeccca01dfa54605b832ff08fdbad525" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeccca01dfa54605b832ff08fdbad525" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34669 } } }
I20260806 15:59:26.537036 31521 sys_catalog.cc:458] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:26.537298 31520 sys_catalog.cc:455] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aeccca01dfa54605b832ff08fdbad525" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aeccca01dfa54605b832ff08fdbad525" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34669 } } }
I20260806 15:59:26.537408 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:59:26.537446 31520 sys_catalog.cc:458] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:26.537762 31528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:26.540757 31528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:26.546010 31528 catalog_manager.cc:1347] Generated new cluster ID: 637c31c697194fc6a49735b187b075d9
I20260806 15:59:26.546074 31528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:26.554934 31528 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:26.555771 31528 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:26.564201 31528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aeccca01dfa54605b832ff08fdbad525: Generated new TSK 0
I20260806 15:59:26.564807 31528 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:59:26.594003 31517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:26.573466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57674 (local address 127.25.31.190:34669)
0806 15:59:26.573575 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:26.573579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:26.573814 (+ 235us) server_negotiation.cc:407] Connection header received
0806 15:59:26.573995 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:26.573999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:26.574054 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:26.574160 (+ 106us) server_negotiation.cc:226] Negotiated authn=JWT
0806 15:59:26.574455 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:26.574992 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:26.575850 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:26.576091 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:26.576200 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:26.576516 (+ 316us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 15:59:26.593527 (+ 17011us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 15:59:26.593845 (+ 318us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57674: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 15:59:26.594094 31517 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57674: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 15:59:26.594141 31543 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:26.573394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:34669 (local address 127.0.0.1:57674)
0806 15:59:26.573724 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:26.573736 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:26.573869 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:26.574163 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:26.574166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:26.574173 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0806 15:59:26.574295 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:26.574340 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:26.575109 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:26.575113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:26.575728 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:26.575735 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:26.576225 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:26.576229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:26.576321 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:26.576343 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 15:59:26.593756 (+ 17413us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 15:59:26.593937 (+ 181us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.31.190:34669: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":107,"threads_started":1}
I20260806 15:59:26.594907 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 31443
2026-08-06T15:59:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (284 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (121 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (122 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:59:26 dist-test-slave-tm1g krb5kdc[31556](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:59:26 dist-test-slave-tm1g krb5kdc[31556](info): set up 2 sockets
Aug 06 15:59:26 dist-test-slave-tm1g krb5kdc[31556](info): commencing operation
krb5kdc: starting...
W20260806 15:59:27.915022 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 15:59:27 dist-test-slave-tm1g krb5kdc[31556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:59:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 15:59:28.035284 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:44343
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:44343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:28.144961 31570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:28.145218 31570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:28.145257 31570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:28.149130 31570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:28.149186 31570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:28.149206 31570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:28.149222 31570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:28.149237 31570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:28.154413 31570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:44343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:44343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:28.155362 31570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:28.156405 31570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:59:28.165083 31570 server_base.cc:1034] running on GCE node
W20260806 15:59:28.165046 31576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:28.165119 31578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:28.165047 31575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:28.165849 31570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:28.166954 31570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:28.168145 31570 hybrid_clock.cc:648] HybridClock initialized: now 1786031968168132 us; error 31 us; skew 500 ppm
Aug 06 15:59:28 dist-test-slave-tm1g krb5kdc[31556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031968, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:28.170840 31570 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 15:59:28.171715 31570 webserver.cc:460] Webserver started at http://127.25.31.190:45909/ using document root <none> and password file <none>
I20260806 15:59:28.172252 31570 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:28.172343 31570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:28.172578 31570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:28.174602 31570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "6ac7257f408c4260b48e121d7900a0a0"
format_stamp: "Formatted at 2026-08-06 15:59:28 on dist-test-slave-tm1g"
server_key: "af0fe36f2483eab49deaaf063b771f37"
server_key_iv: "23f9c239879b4bce5a044068d2d8f043"
server_key_version: "encryptionkey@0"
I20260806 15:59:28.175165 31570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "6ac7257f408c4260b48e121d7900a0a0"
format_stamp: "Formatted at 2026-08-06 15:59:28 on dist-test-slave-tm1g"
server_key: "af0fe36f2483eab49deaaf063b771f37"
server_key_iv: "23f9c239879b4bce5a044068d2d8f043"
server_key_version: "encryptionkey@0"
I20260806 15:59:28.178702 31570 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 15:59:28.181005 31585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:28.182114 31570 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:28.182250 31570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "6ac7257f408c4260b48e121d7900a0a0"
format_stamp: "Formatted at 2026-08-06 15:59:28 on dist-test-slave-tm1g"
server_key: "af0fe36f2483eab49deaaf063b771f37"
server_key_iv: "23f9c239879b4bce5a044068d2d8f043"
server_key_version: "encryptionkey@0"
I20260806 15:59:28.182394 31570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:28.195325 31570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:28.198128 31570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:28.198390 31570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:28.205372 31570 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:44343
I20260806 15:59:28.205430 31640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:44343 every 8 connection(s)
I20260806 15:59:28.206346 31570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:28.208967 31641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:28.210942 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 31570
I20260806 15:59:28.211069 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:28.211392 25726 external_mini_cluster.cc:1428] Setting key 8525c9450ea9c09eb7c0852c115d351d
I20260806 15:59:28.215627 31641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0: Bootstrap starting.
I20260806 15:59:28.218134 31641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:28.219061 31641 log.cc:824] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0: Log is configured to *not* fsync() on all Append() calls
Aug 06 15:59:28 dist-test-slave-tm1g krb5kdc[31556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:28.221069 31641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0: No bootstrap required, opened a new log
I20260806 15:59:28.225940 31644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:28.212707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.25.31.190:44343)
0806 15:59:28.213086 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:28.213103 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:28.213136 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:59:28.213749 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:28.213765 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:28.214065 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:28.214378 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:28.215278 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:28.216292 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:28.217452 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:28.217897 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:28.218018 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:28.220766 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:28.220773 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:28.220788 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:28.220818 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:28.222837 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:28.223277 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:28.223284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:28.223289 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:28.223447 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:28.223765 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:28.223770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:28.223772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:28.224256 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:28.224436 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:28.224718 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 15:59:28.225001 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":123,"threads_started":1}
I20260806 15:59:28.225831 31641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac7257f408c4260b48e121d7900a0a0" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44343 } }
I20260806 15:59:28.226073 31641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:28.226125 31641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ac7257f408c4260b48e121d7900a0a0, State: Initialized, Role: FOLLOWER
I20260806 15:59:28.226608 31641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac7257f408c4260b48e121d7900a0a0" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44343 } }
I20260806 15:59:28.226763 31641 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:28.226827 31641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:28.226922 31641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:28.227759 31641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac7257f408c4260b48e121d7900a0a0" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44343 } }
I20260806 15:59:28.228086 31641 leader_election.cc:302] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ac7257f408c4260b48e121d7900a0a0; no voters:
I20260806 15:59:28.228384 31641 leader_election.cc:288] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:28.228497 31646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:28.228727 31646 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [term 1 LEADER]: Becoming Leader. State: Replica: 6ac7257f408c4260b48e121d7900a0a0, State: Running, Role: LEADER
I20260806 15:59:28.229049 31646 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac7257f408c4260b48e121d7900a0a0" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44343 } }
I20260806 15:59:28.229496 31641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:28.230619 31648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ac7257f408c4260b48e121d7900a0a0. Latest consensus state: current_term: 1 leader_uuid: "6ac7257f408c4260b48e121d7900a0a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac7257f408c4260b48e121d7900a0a0" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44343 } } }
I20260806 15:59:28.230756 31648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:28.231132 31657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:28.231127 31647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ac7257f408c4260b48e121d7900a0a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ac7257f408c4260b48e121d7900a0a0" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44343 } } }
I20260806 15:59:28.231803 31647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:28.234036 31657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:28.239431 31657 catalog_manager.cc:1347] Generated new cluster ID: d3e0c25184574f068fc73b57b27fbfb1
I20260806 15:59:28.239585 31657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:28.282274 31657 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:28.283403 31657 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:28.289440 31657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ac7257f408c4260b48e121d7900a0a0: Generated new TSK 0
I20260806 15:59:28.290098 31657 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 15:59:28.321295 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:44343
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:28.455839 31669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:28.456077 31669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:28.456120 31669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:28.460223 31669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:28.460276 31669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:28.460364 31669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:28.465416 31669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:44343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:28.466387 31669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:28.467353 31669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:28.476449 31677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:28.476549 31669 server_base.cc:1034] running on GCE node
W20260806 15:59:28.476533 31675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:28.476462 31674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:28.477212 31669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:28.477794 31669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:28.478971 31669 hybrid_clock.cc:648] HybridClock initialized: now 1786031968478959 us; error 24 us; skew 500 ppm
Aug 06 15:59:28 dist-test-slave-tm1g krb5kdc[31556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031968, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:28.481599 31669 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 15:59:28.482509 31669 webserver.cc:460] Webserver started at http://127.25.31.129:38603/ using document root <none> and password file <none>
I20260806 15:59:28.483043 31669 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:28.483134 31669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:28.483359 31669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:28.485289 31669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "a77e11fd407541d7b9466623716a620d"
format_stamp: "Formatted at 2026-08-06 15:59:28 on dist-test-slave-tm1g"
server_key: "295f17b13e623bf86ba5e40e4efd327d"
server_key_iv: "887ecb64844b9dc713024aa4e5f5095a"
server_key_version: "encryptionkey@0"
I20260806 15:59:28.485862 31669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "a77e11fd407541d7b9466623716a620d"
format_stamp: "Formatted at 2026-08-06 15:59:28 on dist-test-slave-tm1g"
server_key: "295f17b13e623bf86ba5e40e4efd327d"
server_key_iv: "887ecb64844b9dc713024aa4e5f5095a"
server_key_version: "encryptionkey@0"
I20260806 15:59:28.489372 31669 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 15:59:28.494626 31684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:28.495942 31669 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260806 15:59:28.496075 31669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "a77e11fd407541d7b9466623716a620d"
format_stamp: "Formatted at 2026-08-06 15:59:28 on dist-test-slave-tm1g"
server_key: "295f17b13e623bf86ba5e40e4efd327d"
server_key_iv: "887ecb64844b9dc713024aa4e5f5095a"
server_key_version: "encryptionkey@0"
I20260806 15:59:28.496182 31669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:28.520097 31669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:28.523068 31669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:28.523267 31669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:28.523854 31669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:28.524655 31669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:28.524701 31669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:28.524770 31669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:28.524802 31669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:28.534183 31669 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:43093
I20260806 15:59:28.534211 31806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:43093 every 8 connection(s)
I20260806 15:59:28.535063 31669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:28.538450 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 31669
I20260806 15:59:28.538583 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 15:59:28.538899 25726 external_mini_cluster.cc:1428] Setting key 03753d9b144811d2418fce2464d71857
Aug 06 15:59:28 dist-test-slave-tm1g krb5kdc[31556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031968, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
I20260806 15:59:28.549908 31644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:28.536868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:37959 (local address 127.25.31.190:44343)
0806 15:59:28.537010 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:28.537013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:28.537818 (+ 805us) server_negotiation.cc:407] Connection header received
0806 15:59:28.539299 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:28.539302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:28.539359 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:28.539447 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:28.540846 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:28.541697 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:28.542992 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:28.543409 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:28.543494 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:28.545950 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:28.545955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:28.545960 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:28.545991 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:28.547795 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:28.548318 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:28.548321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:28.548323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:28.548420 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:28.548767 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:28.548773 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:28.548778 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:28.549021 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:28.549128 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:28.549681 (+ 553us) server_negotiation.cc:299] Negotiation successful
0806 15:59:28.549788 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 15:59:28.550818 31809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:28.537237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:44343 (local address 127.25.31.129:37959)
0806 15:59:28.537671 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:28.537711 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:28.539101 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:28.539650 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:28.539658 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:28.540123 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:28.540695 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:28.540709 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:28.541842 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:28.541845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:28.542866 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:28.542875 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:28.543540 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:28.543543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:28.543679 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:28.544274 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:28.544290 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:28.545785 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:28.547928 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:28.547933 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:28.547944 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:28.548212 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:28.548525 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:28.548527 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:28.548529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:28.548662 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:28.549136 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:28.549141 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:28.549363 (+ 222us) client_negotiation.cc:769] Sending connection context
0806 15:59:28.549596 (+ 233us) client_negotiation.cc:240] Negotiation successful
0806 15:59:28.549846 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":118,"threads_started":1}
I20260806 15:59:28.552063 31807 heartbeater.cc:347] Connected to a master server at 127.25.31.190:44343
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
I20260806 15:59:28.552304 31807 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:28.552901 31807 heartbeater.cc:511] Master 127.25.31.190:44343 requested a full tablet report, sending...
I20260806 15:59:28.554683 31602 ts_manager.cc:194] Registered new tserver with Master: a77e11fd407541d7b9466623716a620d (127.25.31.129:43093)
I20260806 15:59:28.556360 31602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:37959
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 15:59:28.567695 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:44343
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:28.675259 31812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:28.675638 31812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:28.675726 31812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:28.679705 31812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:28.679752 31812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:28.679838 31812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:28.685832 31812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:44343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:28.686968 31812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:28.687924 31812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:28.697088 31817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:28.697113 31820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:28.697113 31818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:28.697203 31812 server_base.cc:1034] running on GCE node
I20260806 15:59:28.697695 31812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:28.698282 31812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:28.699502 31812 hybrid_clock.cc:648] HybridClock initialized: now 1786031968699488 us; error 24 us; skew 500 ppm
Aug 06 15:59:28 dist-test-slave-tm1g krb5kdc[31556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031968, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:28.702430 31812 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 15:59:28.703562 31812 webserver.cc:460] Webserver started at http://127.25.31.130:39011/ using document root <none> and password file <none>
I20260806 15:59:28.704104 31812 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:28.704165 31812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:28.704397 31812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:28.706439 31812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "7ab88f0605d14d82b2d9f2321cef81f7"
format_stamp: "Formatted at 2026-08-06 15:59:28 on dist-test-slave-tm1g"
server_key: "472181ed2c823a1e37569d21f6eb10dd"
server_key_iv: "a1e81b749d9387196711de663f643b2b"
server_key_version: "encryptionkey@0"
I20260806 15:59:28.706996 31812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "7ab88f0605d14d82b2d9f2321cef81f7"
format_stamp: "Formatted at 2026-08-06 15:59:28 on dist-test-slave-tm1g"
server_key: "472181ed2c823a1e37569d21f6eb10dd"
server_key_iv: "a1e81b749d9387196711de663f643b2b"
server_key_version: "encryptionkey@0"
I20260806 15:59:28.710512 31812 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 15:59:28.712797 31827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:28.713908 31812 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:28.714030 31812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "7ab88f0605d14d82b2d9f2321cef81f7"
format_stamp: "Formatted at 2026-08-06 15:59:28 on dist-test-slave-tm1g"
server_key: "472181ed2c823a1e37569d21f6eb10dd"
server_key_iv: "a1e81b749d9387196711de663f643b2b"
server_key_version: "encryptionkey@0"
I20260806 15:59:28.714130 31812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:28.732177 31812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:28.734933 31812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:28.735126 31812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:28.735673 31812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:28.736513 31812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:28.736560 31812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:28.736629 31812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:28.736665 31812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:28.745110 31812 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:41203
I20260806 15:59:28.745189 31948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:41203 every 8 connection(s)
I20260806 15:59:28.745992 31812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:28.753420 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 31812
I20260806 15:59:28.753551 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 15:59:28.753842 25726 external_mini_cluster.cc:1428] Setting key 6d0babc706a810341d7cb70bdcc13af7
Aug 06 15:59:28 dist-test-slave-tm1g krb5kdc[31556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031968, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:28.760457 31644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:28.747802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:41843 (local address 127.25.31.190:44343)
0806 15:59:28.747975 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:28.747981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:28.748782 (+ 801us) server_negotiation.cc:407] Connection header received
0806 15:59:28.750043 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:28.750047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:28.750094 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:28.750170 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:28.751320 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:28.751862 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:28.753037 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:28.753421 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:28.753551 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:28.756481 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:28.756488 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:28.756494 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:28.756528 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:28.758107 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:28.758779 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:28.758781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:28.758783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:28.758861 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:28.759287 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:28.759290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:28.759291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:28.759495 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:28.759578 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:28.760227 (+ 649us) server_negotiation.cc:299] Negotiation successful
0806 15:59:28.760341 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 15:59:28.761598 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:28.748167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:44343 (local address 127.25.31.130:41843)
0806 15:59:28.748635 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:28.748675 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:28.749859 (+ 1184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:28.750384 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:28.750393 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:28.750738 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:28.751184 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:28.751196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:28.752000 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:28.752003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:28.752905 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:28.752914 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:28.753628 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:28.753635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:28.753802 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:28.754520 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:28.754535 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:28.756311 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:28.758227 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:28.758236 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:28.758254 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:28.758680 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:28.758953 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:28.758959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:28.758963 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:28.759152 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:28.759608 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:28.759617 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:28.759942 (+ 325us) client_negotiation.cc:769] Sending connection context
0806 15:59:28.760165 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 15:59:28.760498 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":133,"threads_started":1}
I20260806 15:59:28.762842 31949 heartbeater.cc:347] Connected to a master server at 127.25.31.190:44343
I20260806 15:59:28.763082 31949 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:28.763556 31949 heartbeater.cc:511] Master 127.25.31.190:44343 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
I20260806 15:59:28.764628 31602 ts_manager.cc:194] Registered new tserver with Master: 7ab88f0605d14d82b2d9f2321cef81f7 (127.25.31.130:41203)
I20260806 15:59:28.765211 31602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:41843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 15:59:28.781982 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:44343
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:28.890731 31954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:28.890985 31954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:28.891054 31954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:28.895031 31954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:28.895104 31954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:28.895184 31954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:28.900445 31954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:44343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:28.901372 31954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:28.902289 31954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:59:28.911834 31954 server_base.cc:1034] running on GCE node
W20260806 15:59:28.911811 31960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:28.911985 31962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:28.911794 31959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:28.912420 31954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:28.913007 31954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:28.914187 31954 hybrid_clock.cc:648] HybridClock initialized: now 1786031968914170 us; error 30 us; skew 500 ppm
Aug 06 15:59:28 dist-test-slave-tm1g krb5kdc[31556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031968, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:28.916865 31954 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 15:59:28.917677 31954 webserver.cc:460] Webserver started at http://127.25.31.131:32921/ using document root <none> and password file <none>
I20260806 15:59:28.918242 31954 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:28.918357 31954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:28.918592 31954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:28.920528 31954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "b360e9837da643d284358c7d1d830750"
format_stamp: "Formatted at 2026-08-06 15:59:28 on dist-test-slave-tm1g"
server_key: "573eeaf8eba3770f8f2ff50381b4d277"
server_key_iv: "d22826f2407079d0dae678ba268a5c30"
server_key_version: "encryptionkey@0"
I20260806 15:59:28.921080 31954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "b360e9837da643d284358c7d1d830750"
format_stamp: "Formatted at 2026-08-06 15:59:28 on dist-test-slave-tm1g"
server_key: "573eeaf8eba3770f8f2ff50381b4d277"
server_key_iv: "d22826f2407079d0dae678ba268a5c30"
server_key_version: "encryptionkey@0"
I20260806 15:59:28.924712 31954 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 15:59:28.927129 31969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:28.928254 31954 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:28.928383 31954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "b360e9837da643d284358c7d1d830750"
format_stamp: "Formatted at 2026-08-06 15:59:28 on dist-test-slave-tm1g"
server_key: "573eeaf8eba3770f8f2ff50381b4d277"
server_key_iv: "d22826f2407079d0dae678ba268a5c30"
server_key_version: "encryptionkey@0"
I20260806 15:59:28.928491 31954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:28.952324 31954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:28.955252 31954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:28.955448 31954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:28.956013 31954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:28.956809 31954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:28.956879 31954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:28.956944 31954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:28.957000 31954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:28.966272 31954 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:42549
I20260806 15:59:28.966321 32088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:42549 every 8 connection(s)
I20260806 15:59:28.967137 31954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 15:59:28.967765 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 31954
I20260806 15:59:28.967904 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 15:59:28.968191 25726 external_mini_cluster.cc:1428] Setting key 7d14c0d2c1895d25a505df29ab9ef85d
Aug 06 15:59:28 dist-test-slave-tm1g krb5kdc[31556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031968, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:28.980119 31644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:28.968924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:58959 (local address 127.25.31.190:44343)
0806 15:59:28.969075 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:28.969081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:28.970068 (+ 987us) server_negotiation.cc:407] Connection header received
0806 15:59:28.971448 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:28.971450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:28.971498 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:28.971564 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:28.972675 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:28.973205 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:28.974367 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:28.974584 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:28.974663 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:28.976946 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:28.976949 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:28.976951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:28.976980 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:28.978266 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:28.978828 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:28.978830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:28.978831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:28.978905 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:28.979187 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:28.979189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:28.979190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:28.979350 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:28.979424 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:28.979904 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 15:59:28.980005 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 15:59:28.980926 32091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:28.969438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:44343 (local address 127.25.31.131:58959)
0806 15:59:28.969910 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:28.969963 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:28.971255 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:28.971720 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:28.971730 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:28.972099 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:28.972556 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:28.972569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:28.973326 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:28.973329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:28.974219 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:28.974228 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:28.974710 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:28.974713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:28.974848 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:28.975362 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:28.975377 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:28.976799 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:28.978480 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:28.978484 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:28.978494 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:28.978742 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:28.978984 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:28.978986 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:28.978987 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:28.979110 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:28.979467 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:28.979472 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:28.979696 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 15:59:28.979854 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 15:59:28.980086 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":137,"threads_started":1}
I20260806 15:59:28.982069 32089 heartbeater.cc:347] Connected to a master server at 127.25.31.190:44343
I20260806 15:59:28.982290 32089 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:28.982787 32089 heartbeater.cc:511] Master 127.25.31.190:44343 requested a full tablet report, sending...
I20260806 15:59:28.983768 31602 ts_manager.cc:194] Registered new tserver with Master: b360e9837da643d284358c7d1d830750 (127.25.31.131:42549)
I20260806 15:59:28.984282 31602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:58959
I20260806 15:59:28.993858 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/test-admin.keytab.
Aug 06 15:59:29 dist-test-slave-tm1g krb5kdc[31556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:29.011471 25726 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 15:59:29.014454 25726 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/krb5cc' of type FILE
W20260806 15:59:29.014470 32097 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:29.012870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46194 (local address 127.25.31.129:43093)
0806 15:59:29.013146 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:29.013166 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:29.013199 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:59:29.013252 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:29.013257 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:29.013458 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:29.013583 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:29.014217 (+ 634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46194: BlockingRecv error: recv got EOF from 127.0.0.1:46194 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":107,"threads_started":1}
Aug 06 15:59:29 dist-test-slave-tm1g krb5kdc[31556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:29.015493 25726 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 15:59:29 dist-test-slave-tm1g krb5kdc[31556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:29.016405 25726 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 15:59:29 dist-test-slave-tm1g krb5kdc[31556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.129@KRBTEST.COM
I20260806 15:59:29.025770 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:29.017320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46202 (local address 127.25.31.129:43093)
0806 15:59:29.017458 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:29.017462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:29.017563 (+ 101us) server_negotiation.cc:407] Connection header received
0806 15:59:29.017726 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:29.017729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:29.017770 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:29.017836 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:29.018502 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:29.019280 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:29.020099 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:29.020447 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:29.020526 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:29.022753 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:29.022761 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:29.022774 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:29.022796 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:29.024234 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:29.024624 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:29.024629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:29.024630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:29.024704 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:29.024980 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:29.024983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:29.024984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:29.025200 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:29.025366 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:29.025532 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 15:59:29.025649 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 15:59:29.029410 32097 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:29.028537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46216 (local address 127.25.31.129:43093)
0806 15:59:29.028658 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:29.028661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:29.028673 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:59:29.028776 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:29.028778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:29.028810 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:29.028869 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:29.029305 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46216: BlockingRecv error: recv got EOF from 127.0.0.1:46216 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 15:59:29.029523 25726 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/krb5cc' of type FILE
Aug 06 15:59:29 dist-test-slave-tm1g krb5kdc[31556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:29.030627 25726 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 15:59:29 dist-test-slave-tm1g krb5kdc[31556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:29.031502 25726 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 15:59:29 dist-test-slave-tm1g krb5kdc[31556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.129@KRBTEST.COM
I20260806 15:59:29.040452 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:29.032284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46222 (local address 127.25.31.129:43093)
0806 15:59:29.032409 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:29.032413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:29.032504 (+ 91us) server_negotiation.cc:407] Connection header received
0806 15:59:29.032641 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:29.032644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:29.032680 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:29.032769 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:29.033283 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:29.033798 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:29.034643 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:29.034942 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:29.035057 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:29.037483 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:29.037486 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:29.037488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:29.037520 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:29.039003 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:29.039360 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:29.039362 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:29.039363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:29.039441 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:29.039694 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:29.039702 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:29.039704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:29.039870 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:29.039975 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:29.040136 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 15:59:29.040263 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 15:59:29.042841 32097 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:29.041975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46238 (local address 127.25.31.129:43093)
0806 15:59:29.042084 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:29.042088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:29.042212 (+ 124us) server_negotiation.cc:407] Connection header received
0806 15:59:29.042294 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:29.042296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:29.042376 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:29.042442 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:29.042740 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46238: BlockingRecv error: recv got EOF from 127.0.0.1:46238 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 15:59:29.042969 25726 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786031939584994-25726-0/krb5kdc/krb5cc' of type FILE
Aug 06 15:59:29 dist-test-slave-tm1g krb5kdc[31556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:29.043886 25726 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 15:59:29 dist-test-slave-tm1g krb5kdc[31556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:29.044721 25726 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 15:59:29 dist-test-slave-tm1g krb5kdc[31556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.129@KRBTEST.COM
I20260806 15:59:29.052783 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:29.045490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46240 (local address 127.25.31.129:43093)
0806 15:59:29.045604 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:29.045608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:29.045705 (+ 97us) server_negotiation.cc:407] Connection header received
0806 15:59:29.045831 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:29.045834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:29.045876 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:29.045955 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:29.046469 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:29.046947 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:29.047714 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:29.047928 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:29.048031 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:29.050001 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:29.050007 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:29.050009 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:29.050035 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:29.051269 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:29.051670 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:29.051673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:29.051675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:29.051754 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:29.052048 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:29.052051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:29.052053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:29.052202 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:29.052329 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:29.052554 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 15:59:29.052660 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 15:59:29.053718 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 31669
I20260806 15:59:29.062692 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 31812
I20260806 15:59:29.068630 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 31954
I20260806 15:59:29.074525 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 31570
2026-08-06T15:59:29Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2237 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:59:29 dist-test-slave-tm1g krb5kdc[32110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:59:29 dist-test-slave-tm1g krb5kdc[32110](info): set up 2 sockets
Aug 06 15:59:29 dist-test-slave-tm1g krb5kdc[32110](info): commencing operation
krb5kdc: starting...
W20260806 15:59:30.126860 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 15:59:30 dist-test-slave-tm1g krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:59:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:30Z Disabled control of system clock
WARNING: no policy specified for oryx/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.190.keytab.
Entry for principal oryx/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.190.keytab.
I20260806 15:59:30.245761 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:40393
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.190.keytab
--principal=oryx/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:40393
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:30.354929 32123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:30.355222 32123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:30.355304 32123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:30.359261 32123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:30.359346 32123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:30.359385 32123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:30.359439 32123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:30.359491 32123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:30.365343 32123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:40393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.190.keytab
--principal=oryx/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:40393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:30.366421 32123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:30.367439 32123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:30.375854 32131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:30.375874 32129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:30.375874 32128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:30.376179 32123 server_base.cc:1034] running on GCE node
I20260806 15:59:30.376713 32123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:30.377722 32123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:30.378913 32123 hybrid_clock.cc:648] HybridClock initialized: now 1786031970378877 us; error 40 us; skew 500 ppm
Aug 06 15:59:30 dist-test-slave-tm1g krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031970, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:30.386430 32123 init.cc:377] Logged in from keytab as oryx/127.25.31.190@KRBTEST.COM (short username oryx)
I20260806 15:59:30.387255 32123 webserver.cc:460] Webserver started at http://127.25.31.190:45919/ using document root <none> and password file <none>
I20260806 15:59:30.387784 32123 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:30.387873 32123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:30.388103 32123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:30.390497 32123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "13d4ef44d33d4e888756a75470614006"
format_stamp: "Formatted at 2026-08-06 15:59:30 on dist-test-slave-tm1g"
server_key: "a756b56f64b9c57fa561bbc4288a6d33"
server_key_iv: "4b2a314ddca40c7b6b59320473af1529"
server_key_version: "encryptionkey@0"
I20260806 15:59:30.391052 32123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "13d4ef44d33d4e888756a75470614006"
format_stamp: "Formatted at 2026-08-06 15:59:30 on dist-test-slave-tm1g"
server_key: "a756b56f64b9c57fa561bbc4288a6d33"
server_key_iv: "4b2a314ddca40c7b6b59320473af1529"
server_key_version: "encryptionkey@0"
I20260806 15:59:30.394615 32123 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 15:59:30.396931 32138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:30.398097 32123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:30.398228 32123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "13d4ef44d33d4e888756a75470614006"
format_stamp: "Formatted at 2026-08-06 15:59:30 on dist-test-slave-tm1g"
server_key: "a756b56f64b9c57fa561bbc4288a6d33"
server_key_iv: "4b2a314ddca40c7b6b59320473af1529"
server_key_version: "encryptionkey@0"
I20260806 15:59:30.398355 32123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:30.422102 32123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:30.425024 32123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:30.425226 32123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:30.432072 32123 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:40393
I20260806 15:59:30.432376 32193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:40393 every 8 connection(s)
I20260806 15:59:30.433024 32123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:30.435698 32194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:30.440932 32194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006: Bootstrap starting.
I20260806 15:59:30.441959 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 32123
I20260806 15:59:30.442104 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:30.442456 25726 external_mini_cluster.cc:1428] Setting key 8d7c9f454e93ef558f4b91ee02a04719
I20260806 15:59:30.443778 32194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:30.444501 32194 log.cc:824] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:30.446725 32194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006: No bootstrap required, opened a new log
Aug 06 15:59:30 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
I20260806 15:59:30.451084 32194 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } }
I20260806 15:59:30.451318 32194 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:30.451390 32194 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13d4ef44d33d4e888756a75470614006, State: Initialized, Role: FOLLOWER
I20260806 15:59:30.451855 32194 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } }
I20260806 15:59:30.452025 32194 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:30.452093 32194 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:30.452208 32194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:30.453241 32194 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } }
I20260806 15:59:30.453583 32194 leader_election.cc:302] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13d4ef44d33d4e888756a75470614006; no voters:
I20260806 15:59:30.453928 32194 leader_election.cc:288] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:30.453962 32199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:30.454138 32199 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [term 1 LEADER]: Becoming Leader. State: Replica: 13d4ef44d33d4e888756a75470614006, State: Running, Role: LEADER
I20260806 15:59:30.454521 32199 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } }
I20260806 15:59:30.455174 32194 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:30.456468 32201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } }
I20260806 15:59:30.456485 32200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13d4ef44d33d4e888756a75470614006. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } }
I20260806 15:59:30.456619 32201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:30.456622 32200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:30.457013 32208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:30.458209 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:30.443545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57564 (local address 127.25.31.190:40393)
0806 15:59:30.443927 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:30.443941 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:30.443976 (+ 35us) server_negotiation.cc:407] Connection header received
0806 15:59:30.444575 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:30.444590 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:30.444987 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:30.445345 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:30.446213 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:30.447281 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:30.448440 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:30.448886 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:30.448996 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:30.451871 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:30.451876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:30.451888 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:30.451913 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:30.453686 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:30.454149 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:30.454156 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:30.454163 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:30.454297 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:30.454605 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:30.454610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:30.454612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:30.455051 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:30.455279 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:30.456971 (+ 1692us) server_negotiation.cc:299] Negotiation successful
0806 15:59:30.457285 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":122,"threads_started":1}
I20260806 15:59:30.460245 32208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:30.465888 32208 catalog_manager.cc:1347] Generated new cluster ID: 4eeb26800d1d43969bcf8b6df3538011
I20260806 15:59:30.465947 32208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:30.471558 32208 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:30.472343 32208 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:30.489009 32208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006: Generated new TSK 0
I20260806 15:59:30.489624 32208 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.129.keytab.
Entry for principal oryx/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.129.keytab.
I20260806 15:59:30.518253 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:40393
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.129.keytab
--principal=oryx/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:30.625427 32220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:30.625655 32220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:30.625696 32220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:30.629593 32220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:30.629645 32220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:30.629699 32220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:30.634615 32220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.129.keytab
--principal=oryx/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:40393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:30.635459 32220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:30.636329 32220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:30.645364 32226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:30.645406 32225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:30.645560 32220 server_base.cc:1034] running on GCE node
W20260806 15:59:30.645349 32228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:30.646036 32220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:30.646621 32220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:30.647835 32220 hybrid_clock.cc:648] HybridClock initialized: now 1786031970647821 us; error 27 us; skew 500 ppm
Aug 06 15:59:30 dist-test-slave-tm1g krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031970, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:30.650413 32220 init.cc:377] Logged in from keytab as oryx/127.25.31.129@KRBTEST.COM (short username oryx)
I20260806 15:59:30.651185 32220 webserver.cc:460] Webserver started at http://127.25.31.129:39047/ using document root <none> and password file <none>
I20260806 15:59:30.651701 32220 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:30.651786 32220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:30.652014 32220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:30.653900 32220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b"
format_stamp: "Formatted at 2026-08-06 15:59:30 on dist-test-slave-tm1g"
server_key: "d81d8bd1a28da9c2765ab227113e47a1"
server_key_iv: "a44a1132fd98f668f5960b75c4b42806"
server_key_version: "encryptionkey@0"
I20260806 15:59:30.654541 32220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b"
format_stamp: "Formatted at 2026-08-06 15:59:30 on dist-test-slave-tm1g"
server_key: "d81d8bd1a28da9c2765ab227113e47a1"
server_key_iv: "a44a1132fd98f668f5960b75c4b42806"
server_key_version: "encryptionkey@0"
I20260806 15:59:30.658123 32220 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 15:59:30.660456 32236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:30.661657 32220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:30.661784 32220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b"
format_stamp: "Formatted at 2026-08-06 15:59:30 on dist-test-slave-tm1g"
server_key: "d81d8bd1a28da9c2765ab227113e47a1"
server_key_iv: "a44a1132fd98f668f5960b75c4b42806"
server_key_version: "encryptionkey@0"
I20260806 15:59:30.661888 32220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:30.671279 32220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:30.673810 32220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:30.673992 32220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:30.674549 32220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:30.675297 32220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:30.675361 32220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:30.675428 32220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:30.675467 32220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:30.684266 32220 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:41197
I20260806 15:59:30.684303 32355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:41197 every 8 connection(s)
I20260806 15:59:30.685070 32220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:30.693729 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 32220
I20260806 15:59:30.693859 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 15:59:30.694207 25726 external_mini_cluster.cc:1428] Setting key f237a1fb88a783e85c70980d3b146d8b
Aug 06 15:59:30 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031970, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
I20260806 15:59:30.700438 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:30.686764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:47987 (local address 127.25.31.190:40393)
0806 15:59:30.686896 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:30.686900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:30.687659 (+ 759us) server_negotiation.cc:407] Connection header received
0806 15:59:30.688958 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:30.688962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:30.689022 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:30.689121 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:30.690205 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:30.690829 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:30.692552 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:30.692973 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:30.693102 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:30.696664 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:30.696666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:30.696668 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:30.696691 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:30.698190 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:30.698860 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:30.698864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:30.698866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:30.698952 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:30.699333 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:30.699336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:30.699338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:30.699537 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:30.699653 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:30.700125 (+ 472us) server_negotiation.cc:299] Negotiation successful
0806 15:59:30.700257 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 15:59:30.701198 32358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:30.687103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.129:47987)
0806 15:59:30.687517 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:30.687558 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:30.688755 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:30.689278 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:30.689284 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:30.689608 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:30.690043 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:30.690055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:30.690951 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:30.690955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:30.692421 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:30.692432 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:30.693787 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:30.693793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:30.693996 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:30.694658 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:30.694680 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:30.696503 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:30.698330 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:30.698339 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:30.698353 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:30.698747 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:30.699050 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:30.699053 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:30.699055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:30.699234 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:30.699647 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:30.699653 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:30.699898 (+ 245us) client_negotiation.cc:769] Sending connection context
0806 15:59:30.700080 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 15:59:30.700383 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":106,"threads_started":1}
I20260806 15:59:30.702644 32356 heartbeater.cc:347] Connected to a master server at 127.25.31.190:40393
I20260806 15:59:30.702890 32356 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:30.703346 32356 heartbeater.cc:511] Master 127.25.31.190:40393 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.25.31.130@KRBTEST.COM; defaulting to no policy
I20260806 15:59:30.704983 32156 ts_manager.cc:194] Registered new tserver with Master: dfad7b8c48ec447ca43f3dc53e5cf34b (127.25.31.129:41197)
I20260806 15:59:30.706125 32156 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:47987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.31.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.130.keytab.
Entry for principal oryx/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.130.keytab.
I20260806 15:59:30.720969 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:40393
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.130.keytab
--principal=oryx/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:30.830468 32361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:30.830704 32361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:30.830787 32361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:30.834765 32361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:30.834821 32361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:30.834909 32361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:30.839932 32361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.130.keytab
--principal=oryx/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:40393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:30.840925 32361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:30.842096 32361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:30.851464 32367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:30.851519 32361 server_base.cc:1034] running on GCE node
W20260806 15:59:30.851472 32369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:30.851738 32366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:30.852137 32361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:30.852743 32361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:30.853924 32361 hybrid_clock.cc:648] HybridClock initialized: now 1786031970853912 us; error 31 us; skew 500 ppm
Aug 06 15:59:30 dist-test-slave-tm1g krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031970, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:30.856859 32361 init.cc:377] Logged in from keytab as oryx/127.25.31.130@KRBTEST.COM (short username oryx)
I20260806 15:59:30.857748 32361 webserver.cc:460] Webserver started at http://127.25.31.130:39591/ using document root <none> and password file <none>
I20260806 15:59:30.858248 32361 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:30.858361 32361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:30.858546 32361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:30.860477 32361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "1d0f242a13004c89911dcd52f996080d"
format_stamp: "Formatted at 2026-08-06 15:59:30 on dist-test-slave-tm1g"
server_key: "4ecf8bddbafda8b15d616c20171d4149"
server_key_iv: "fe81bf67dbe3671aa2a165e9d312ec90"
server_key_version: "encryptionkey@0"
I20260806 15:59:30.860978 32361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "1d0f242a13004c89911dcd52f996080d"
format_stamp: "Formatted at 2026-08-06 15:59:30 on dist-test-slave-tm1g"
server_key: "4ecf8bddbafda8b15d616c20171d4149"
server_key_iv: "fe81bf67dbe3671aa2a165e9d312ec90"
server_key_version: "encryptionkey@0"
I20260806 15:59:30.864619 32361 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 15:59:30.866964 32376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:30.868222 32361 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:30.868352 32361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "1d0f242a13004c89911dcd52f996080d"
format_stamp: "Formatted at 2026-08-06 15:59:30 on dist-test-slave-tm1g"
server_key: "4ecf8bddbafda8b15d616c20171d4149"
server_key_iv: "fe81bf67dbe3671aa2a165e9d312ec90"
server_key_version: "encryptionkey@0"
I20260806 15:59:30.868456 32361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:30.895255 32361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:30.898159 32361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:30.898391 32361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:30.898993 32361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:30.899820 32361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:30.899868 32361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:30.899930 32361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:30.899971 32361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:30.908869 32361 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:43327
I20260806 15:59:30.908914 32496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:43327 every 8 connection(s)
I20260806 15:59:30.909726 32361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:30.917030 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 32361
I20260806 15:59:30.917155 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 15:59:30.917449 25726 external_mini_cluster.cc:1428] Setting key 64e5a1f790d7829b774b460a3d376b63
Aug 06 15:59:30 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031970, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
I20260806 15:59:30.924032 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:30.911407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:59397 (local address 127.25.31.190:40393)
0806 15:59:30.911547 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:30.911556 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:30.912335 (+ 779us) server_negotiation.cc:407] Connection header received
0806 15:59:30.913690 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:30.913694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:30.913755 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:30.913834 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:30.915033 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:30.915582 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:30.916717 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:30.916959 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:30.917071 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:30.919800 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:30.919804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:30.919810 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:30.919845 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:30.921534 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:30.922202 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:30.922208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:30.922213 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:30.922363 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:30.922700 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:30.922707 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:30.922711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:30.922967 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:30.923072 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:30.923681 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 15:59:30.923832 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 15:59:30.924741 32499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:30.911736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.130:59397)
0806 15:59:30.912168 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:30.912211 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:30.913484 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:30.913997 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:30.914003 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:30.914413 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:30.914885 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:30.914899 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:30.915716 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:30.915720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:30.916592 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:30.916602 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:30.917116 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:30.917121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:30.917320 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:30.917976 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:30.917999 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:30.919619 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:30.921658 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:30.921665 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:30.921684 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:30.922080 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:30.922469 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:30.922471 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:30.922473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:30.922607 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:30.923080 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:30.923084 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:30.923302 (+ 218us) client_negotiation.cc:769] Sending connection context
0806 15:59:30.923605 (+ 303us) client_negotiation.cc:240] Negotiation successful
0806 15:59:30.923879 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":120,"threads_started":1}
I20260806 15:59:30.925874 32497 heartbeater.cc:347] Connected to a master server at 127.25.31.190:40393
I20260806 15:59:30.926105 32497 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:30.926621 32497 heartbeater.cc:511] Master 127.25.31.190:40393 requested a full tablet report, sending...
I20260806 15:59:30.927692 32156 ts_manager.cc:194] Registered new tserver with Master: 1d0f242a13004c89911dcd52f996080d (127.25.31.130:43327)
WARNING: no policy specified for oryx/127.25.31.131@KRBTEST.COM; defaulting to no policy
I20260806 15:59:30.928253 32156 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:59397
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.31.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.131.keytab.
Entry for principal oryx/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.131.keytab.
I20260806 15:59:30.945716 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:40393
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.131.keytab
--principal=oryx/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:31.054616 32502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:31.054920 32502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:31.055004 32502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:31.059126 32502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:31.059201 32502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:31.059288 32502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:31.064477 32502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.131.keytab
--principal=oryx/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:40393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:31.065518 32502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:31.066529 32502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:31.075702 32507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:31.075730 32510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:31.075721 32508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:31.076016 32502 server_base.cc:1034] running on GCE node
I20260806 15:59:31.076376 32502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:31.076974 32502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:31.078128 32502 hybrid_clock.cc:648] HybridClock initialized: now 1786031971078112 us; error 32 us; skew 500 ppm
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031971, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:31.080935 32502 init.cc:377] Logged in from keytab as oryx/127.25.31.131@KRBTEST.COM (short username oryx)
I20260806 15:59:31.081786 32502 webserver.cc:460] Webserver started at http://127.25.31.131:40117/ using document root <none> and password file <none>
I20260806 15:59:31.082373 32502 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:31.082455 32502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:31.082665 32502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:31.084584 32502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "8b17d836eb794e359e1e0ceecb09f970"
format_stamp: "Formatted at 2026-08-06 15:59:31 on dist-test-slave-tm1g"
server_key: "fc44bd11e85f1b3c996c93a61afc0910"
server_key_iv: "5292cb90a84f298e3e49c5c7facf10ef"
server_key_version: "encryptionkey@0"
I20260806 15:59:31.085136 32502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "8b17d836eb794e359e1e0ceecb09f970"
format_stamp: "Formatted at 2026-08-06 15:59:31 on dist-test-slave-tm1g"
server_key: "fc44bd11e85f1b3c996c93a61afc0910"
server_key_iv: "5292cb90a84f298e3e49c5c7facf10ef"
server_key_version: "encryptionkey@0"
I20260806 15:59:31.088761 32502 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 15:59:31.091056 32517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:31.092139 32502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:31.092260 32502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "8b17d836eb794e359e1e0ceecb09f970"
format_stamp: "Formatted at 2026-08-06 15:59:31 on dist-test-slave-tm1g"
server_key: "fc44bd11e85f1b3c996c93a61afc0910"
server_key_iv: "5292cb90a84f298e3e49c5c7facf10ef"
server_key_version: "encryptionkey@0"
I20260806 15:59:31.092356 32502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:31.121717 32502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:31.124677 32502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:31.124895 32502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:31.125507 32502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:31.126344 32502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:31.126390 32502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:31.126457 32502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:31.126498 32502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:31.135110 32502 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:37269
I20260806 15:59:31.135138 32634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:37269 every 8 connection(s)
I20260806 15:59:31.135958 32502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 15:59:31.143658 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 32502
I20260806 15:59:31.143793 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 15:59:31.144176 25726 external_mini_cluster.cc:1428] Setting key d66e973bc2753116b346b98c30d6233a
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031971, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
I20260806 15:59:31.149214 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.137653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:49053 (local address 127.25.31.190:40393)
0806 15:59:31.137791 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.137795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.138622 (+ 827us) server_negotiation.cc:407] Connection header received
0806 15:59:31.139937 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.139940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.139999 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.140103 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.141214 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.141751 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.142894 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.143124 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.143226 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.145908 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.145912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.145914 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.145946 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.147296 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.147779 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.147783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.147785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.147867 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.148181 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.148186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.148187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.148361 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.148463 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.148914 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.149039 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 15:59:31.149917 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.138010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.131:49053)
0806 15:59:31.138476 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.138526 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.139731 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.140272 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.140278 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.140622 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.141067 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.141079 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.141873 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.141876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.142771 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.142781 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.143276 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.143280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.143484 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.144132 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:31.144153 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:31.145754 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:31.147407 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.147413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.147425 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.147678 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.147953 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.147957 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.147959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.148090 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.148462 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:31.148468 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:31.148697 (+ 229us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.148872 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.149172 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":110,"threads_started":1}
I20260806 15:59:31.150933 32635 heartbeater.cc:347] Connected to a master server at 127.25.31.190:40393
I20260806 15:59:31.151129 32635 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:31.151535 32635 heartbeater.cc:511] Master 127.25.31.190:40393 requested a full tablet report, sending...
I20260806 15:59:31.152446 32156 ts_manager.cc:194] Registered new tserver with Master: 8b17d836eb794e359e1e0ceecb09f970 (127.25.31.131:37269)
I20260806 15:59:31.152952 32156 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:49053
I20260806 15:59:31.157910 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.25.31.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.31.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.31.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.189.keytab.
Entry for principal oryx/127.25.31.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.189.keytab.
I20260806 15:59:31.184706 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.189:32931
--webserver_interface=127.25.31.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.189.keytab
--principal=oryx/127.25.31.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:40393,127.25.31.189:32931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:31.292294 32640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:31.292524 32640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:31.292601 32640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:31.296818 32640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:31.296875 32640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:31.296921 32640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:31.296963 32640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:31.296990 32640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:31.302186 32640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:40393,127.25.31.189:32931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.189.keytab
--principal=oryx/127.25.31.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.189:32931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.31.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
W20260806 15:59:31.302424 32640 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.25.31.190:40393,127.25.31.189:32931'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 15:59:31.303282 32640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:31.304255 32640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:31.312196 32646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:31.312202 32645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:31.312489 32640 server_base.cc:1034] running on GCE node
W20260806 15:59:31.312202 32648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:31.313086 32640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:31.314110 32640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:31.315285 32640 hybrid_clock.cc:648] HybridClock initialized: now 1786031971315257 us; error 39 us; skew 500 ppm
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031971, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:31.317929 32640 init.cc:377] Logged in from keytab as oryx/127.25.31.189@KRBTEST.COM (short username oryx)
I20260806 15:59:31.318837 32640 webserver.cc:460] Webserver started at http://127.25.31.189:40027/ using document root <none> and password file <none>
I20260806 15:59:31.319392 32640 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:31.319479 32640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:31.319697 32640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:31.321688 32640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/data/instance:
uuid: "7bb3bdf4e8a24987b424f27bc1daa914"
format_stamp: "Formatted at 2026-08-06 15:59:31 on dist-test-slave-tm1g"
server_key: "be6d63cb795a637209f1957c46a25942"
server_key_iv: "d8e88909261371e823e6b0097de852e1"
server_key_version: "encryptionkey@0"
I20260806 15:59:31.322252 32640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/wal/instance:
uuid: "7bb3bdf4e8a24987b424f27bc1daa914"
format_stamp: "Formatted at 2026-08-06 15:59:31 on dist-test-slave-tm1g"
server_key: "be6d63cb795a637209f1957c46a25942"
server_key_iv: "d8e88909261371e823e6b0097de852e1"
server_key_version: "encryptionkey@0"
I20260806 15:59:31.325687 32640 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 15:59:31.327951 32657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:31.329033 32640 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 15:59:31.329164 32640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/wal
uuid: "7bb3bdf4e8a24987b424f27bc1daa914"
format_stamp: "Formatted at 2026-08-06 15:59:31 on dist-test-slave-tm1g"
server_key: "be6d63cb795a637209f1957c46a25942"
server_key_iv: "d8e88909261371e823e6b0097de852e1"
server_key_version: "encryptionkey@0"
I20260806 15:59:31.329274 32640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:31.348150 32640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:31.351220 32640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:31.351419 32640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:31.358163 32640 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.189:32931
I20260806 15:59:31.358464 32711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.189:32931 every 8 connection(s)
I20260806 15:59:31.359143 32640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/data/info.pb
I20260806 15:59:31.359983 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 32640
I20260806 15:59:31.361955 32712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:31.366348 32712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.31.189@KRBTEST.COM
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031971, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.189@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
I20260806 15:59:31.374974 32716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.361435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32818 (local address 127.25.31.189:32931)
0806 15:59:31.361809 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.361827 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.361871 (+ 44us) server_negotiation.cc:407] Connection header received
0806 15:59:31.362599 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.362617 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.362984 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.363247 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.364109 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.365199 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.366366 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.366936 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.367051 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.369748 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.369752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.369760 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.369784 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.371882 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.372356 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.372365 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.372373 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.372500 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.372802 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.372807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.372812 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.373284 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.373468 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.373875 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.374140 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":132,"threads_started":1}
I20260806 15:59:31.377460 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.368184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57572 (local address 127.25.31.190:40393)
0806 15:59:31.368333 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.368338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.368574 (+ 236us) server_negotiation.cc:407] Connection header received
0806 15:59:31.368750 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.368756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.368817 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.368906 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.369646 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.370337 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.371694 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.372011 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.372120 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.374375 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.374378 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.374380 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.374402 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.375910 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.376332 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.376336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.376338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.376425 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.376676 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.376679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.376682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.376909 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.377007 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.377260 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.377350 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 15:59:31.377465 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.368111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:57572)
0806 15:59:31.368433 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.368480 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.368595 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.368985 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.368989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.369384 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.369515 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.369521 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.370493 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.370500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.371570 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.371585 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.372124 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.372126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.372215 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.372695 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:31.372706 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:31.374204 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:31.376025 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.376033 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.376036 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.376241 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.376512 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.376514 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.376515 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.376593 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.377013 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:31.377017 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:31.377107 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.377213 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.377334 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":94,"threads_started":1}
I20260806 15:59:31.378681 32712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031971, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.189@KRBTEST.COM for oryx/127.25.31.189@KRBTEST.COM
I20260806 15:59:31.385471 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.379106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:32931 (local address 127.0.0.1:32830)
0806 15:59:31.379187 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.379201 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.379267 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.379442 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.379444 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.379572 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.379654 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.379658 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.380690 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.380692 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.381167 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.381174 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.381554 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.381557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.381625 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.381952 (+ 327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:31.381964 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:31.383016 (+ 1052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:31.384330 (+ 1314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.384332 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.384333 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.384514 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.384737 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.384739 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.384740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.384810 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.385096 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:31.385098 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:31.385165 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.385241 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.385352 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 15:59:31.385535 32716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.379154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32830 (local address 127.25.31.189:32931)
0806 15:59:31.379260 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.379263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.379274 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:59:31.379327 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.379329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.379359 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.379419 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.379738 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.380594 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.381253 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.381465 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.381536 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.383127 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.383130 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.383132 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.383151 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.384242 (+ 1091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.384600 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.384602 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.384603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.384667 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.384877 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.384879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.384880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.385023 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.385091 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.385289 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.385404 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":38}
I20260806 15:59:31.388087 32712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: Bootstrap starting.
I20260806 15:59:31.390169 32712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:31.390892 32712 log.cc:824] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:31.392539 32712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: No bootstrap required, opened a new log
I20260806 15:59:31.395465 32712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } }
I20260806 15:59:31.395638 32712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:31.395713 32712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb3bdf4e8a24987b424f27bc1daa914, State: Initialized, Role: FOLLOWER
I20260806 15:59:31.396116 32712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } }
I20260806 15:59:31.396597 32719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } } }
I20260806 15:59:31.396727 32719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:59:31.397223 32712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 15:59:31.401305 32731 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:59:31.401392 32731 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:59:31.403609 32640 master_runner.cc:428] Detected that this master 7bb3bdf4e8a24987b424f27bc1daa914 is joining an existing cluster
I20260806 15:59:31.403671 32640 master_runner.cc:432] Initiating AddMaster RPC to add 127.25.31.189:32931
I20260806 15:59:31.414076 32716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.405965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32836 (local address 127.25.31.189:32931)
0806 15:59:31.406070 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.406075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.406087 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:59:31.406121 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.406124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.406163 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.406244 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.406822 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.407592 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.409259 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.409581 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.409686 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.410937 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.410941 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.410945 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.410966 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.412479 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.412925 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.412928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.412930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.413020 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.413281 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.413284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.413285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.413448 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.413550 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.413775 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.413920 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":44}
I20260806 15:59:31.414090 32738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.405643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:32931 (local address 127.0.0.1:32836)
0806 15:59:31.405917 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.405933 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.406027 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.406257 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.406263 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.406558 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.406696 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.406705 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.407702 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.407705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.408577 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.408583 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.409694 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.409697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.409801 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.410236 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:31.410251 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:31.410798 (+ 547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:31.412600 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.412604 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.412606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.412796 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.413108 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.413110 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.413112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.413201 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.413538 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:31.413541 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:31.413620 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.413732 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.413870 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":47,"thread_start_us":106,"threads_started":1}
I20260806 15:59:31.414489 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.405628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57576 (local address 127.25.31.190:40393)
0806 15:59:31.405752 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.405757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.405999 (+ 242us) server_negotiation.cc:407] Connection header received
0806 15:59:31.406163 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.406166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.406208 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.406278 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.406908 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.407416 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.408682 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.409041 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.409149 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.411077 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.411081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.411084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.411108 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.412774 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.413179 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.413183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.413184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.413279 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.413575 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.413578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.413580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.413779 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.413848 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.414199 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.414339 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 15:59:31.414963 32737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.405578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:57576)
0806 15:59:31.405917 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.405933 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.406048 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.406436 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.406441 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.406686 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.406818 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.406824 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.407523 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.407528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.408557 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.408568 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.409160 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.409162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.409227 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.410505 (+ 1278us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:31.410516 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:31.410941 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:31.412887 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.412892 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.412895 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.413100 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.413383 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.413386 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.413388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.413475 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.413918 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:31.413920 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:31.414023 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.414666 (+ 643us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.414796 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":51,"thread_start_us":83,"threads_started":1}
I20260806 15:59:31.416383 32719 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:31.416520 32719 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } }
I20260806 15:59:31.417374 32719 leader_election.cc:288] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13d4ef44d33d4e888756a75470614006 (127.25.31.190:40393)
I20260806 15:59:31.417804 32167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13d4ef44d33d4e888756a75470614006" is_pre_election: true
I20260806 15:59:31.418427 32659 leader_election.cc:302] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 7bb3bdf4e8a24987b424f27bc1daa914; no voters: 13d4ef44d33d4e888756a75470614006
I20260806 15:59:31.418694 32719 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031970, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.190@KRBTEST.COM for oryx/127.25.31.189@KRBTEST.COM
I20260806 15:59:31.428069 32716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.419589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32848 (local address 127.25.31.189:32931)
0806 15:59:31.419687 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.419692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.419972 (+ 280us) server_negotiation.cc:407] Connection header received
0806 15:59:31.420164 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.420167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.420211 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.420295 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.420892 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.421400 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.422257 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.422518 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.422616 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.424835 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.424839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.424840 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.424862 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.426218 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.426694 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.426699 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.426701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.426775 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.427045 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.427050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.427051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.427255 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.427366 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.427708 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.427932 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 15:59:31.428232 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.419560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:32931 (local address 127.0.0.1:32848)
0806 15:59:31.419862 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.419899 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.420039 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.420327 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.420333 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.420666 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.420767 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.420773 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.421506 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.421511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.422134 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.422143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.422630 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.422634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.422751 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.423206 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:31.423217 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:31.424695 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:31.426347 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.426360 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.426366 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.426593 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.426871 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.426873 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.426875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.426959 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.427373 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:31.427379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:31.427500 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.427649 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.428058 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":88,"threads_started":1}
I20260806 15:59:31.429617 32156 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.25.31.189:32931: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914"
member_type: NON_VOTER
last_known_addr {
host: "127.25.31.189"
port: 32931
}
attrs {
promote: true
}
}
dest_uuid: "13d4ef44d33d4e888756a75470614006"
cas_config_opid_index: -1
I20260806 15:59:31.430258 32156 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: NON_VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: true } }
I20260806 15:59:31.432583 32684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:31.434278 32684 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 FOLLOWER]: Refusing update from remote peer 13d4ef44d33d4e888756a75470614006: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 15:59:31.435196 32741 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7bb3bdf4e8a24987b424f27bc1daa914 (127.25.31.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: NON_VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: true } } }
I20260806 15:59:31.437362 32741 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.25.31.189:32931
I20260806 15:59:31.437934 32640 master.cc:537] Master@127.25.31.189:32931 shutting down...
WARNING: no policy specified for oryx/127.25.31.188@KRBTEST.COM; defaulting to no policy
I20260806 15:59:31.438344 32741 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: NON_VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 15:59:31.440217 32139 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 15:59:31.440590 32741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: NON_VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: true } } }
I20260806 15:59:31.440688 32741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.31.188@KRBTEST.COM" created.
W20260806 15:59:31.441229 32139 consensus_peers.cc:487] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 -> Peer 7bb3bdf4e8a24987b424f27bc1daa914 (127.25.31.189:32931): Couldn't send request to peer 7bb3bdf4e8a24987b424f27bc1daa914. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 15:59:31.443248 32640 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:31.443528 32640 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:31.443626 32640 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: stopping tablet replica
I20260806 15:59:31.447786 32640 master.cc:552] Master@127.25.31.189:32931 shutdown complete.
I20260806 15:59:31.447850 32640 master_runner.cc:305] Clearing existing system tablet
I20260806 15:59:31.448067 32640 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:31.448139 32640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:31.449245 32640 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:59:31.450050 32748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:31.450248 32640 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:31.450417 32640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:59:31.450474 32640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/wal
uuid: "7bb3bdf4e8a24987b424f27bc1daa914"
format_stamp: "Formatted at 2026-08-06 15:59:31 on dist-test-slave-tm1g"
server_key: "be6d63cb795a637209f1957c46a25942"
server_key_iv: "d8e88909261371e823e6b0097de852e1"
server_key_version: "encryptionkey@0"
I20260806 15:59:31.452003 32640 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:31.453912 32640 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 15:59:31.454015 32640 log.cc:1196] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 15:59:31.454337 32640 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: Deleting consensus metadata
I20260806 15:59:31.454547 32640 master_runner.cc:315] Copying system tablet from 127.25.31.190:40393
I20260806 15:59:31.455715 32640 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: tablet copy: Beginning tablet copy session from remote peer at address 127.25.31.190:40393
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.31.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.188.keytab.
Entry for principal oryx/127.25.31.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.188.keytab.
I20260806 15:59:31.465286 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.188:32771
--webserver_interface=127.25.31.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.188.keytab
--principal=oryx/127.25.31.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:40393,127.25.31.189:32931,127.25.31.188:32771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:59:31.466734 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.456291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57582 (local address 127.25.31.190:40393)
0806 15:59:31.456425 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.456430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.458674 (+ 2244us) server_negotiation.cc:407] Connection header received
0806 15:59:31.458791 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.458794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.458838 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.458913 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.459409 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.460061 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.461196 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.461455 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.461540 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.462911 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.462915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.462917 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.462972 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.464777 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.465385 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.465389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.465394 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.465499 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.465791 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.465794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.465796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.466037 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.466134 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.466467 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.466574 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 15:59:31.466743 32754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.456243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:57582)
0806 15:59:31.458577 (+ 2334us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.458591 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.458705 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.458962 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.458965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.459175 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.459299 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.459306 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.460173 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.460176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.461074 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.461084 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.461565 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.461569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.461665 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.462278 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:31.462293 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:31.462750 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:31.465058 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.465061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.465063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.465251 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.465598 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.465600 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.465604 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.465695 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.466125 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:31.466128 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:31.466231 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.466399 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.466545 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2252,"thread_start_us":136,"threads_started":1}
I20260806 15:59:31.467480 32178 tablet_copy_service.cc:140] P 13d4ef44d33d4e888756a75470614006: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7bb3bdf4e8a24987b424f27bc1daa914 ({username='oryx', principal='oryx/127.25.31.189@KRBTEST.COM'} at 127.0.0.1:57582)
I20260806 15:59:31.467626 32178 tablet_copy_service.cc:161] P 13d4ef44d33d4e888756a75470614006: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7bb3bdf4e8a24987b424f27bc1daa914 at {username='oryx', principal='oryx/127.25.31.189@KRBTEST.COM'} at 127.0.0.1:57582: session id = 7bb3bdf4e8a24987b424f27bc1daa914-00000000000000000000000000000000
I20260806 15:59:31.469326 32178 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 15:59:31.470829 32640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 15:59:31.472891 32640 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: tablet copy: Starting download of 0 data blocks...
I20260806 15:59:31.473090 32640 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: tablet copy: Starting download of 1 WAL segments...
I20260806 15:59:31.474655 32640 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 15:59:31.476068 32178 tablet_copy_service.cc:342] P 13d4ef44d33d4e888756a75470614006: Request end of tablet copy session 7bb3bdf4e8a24987b424f27bc1daa914-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.25.31.189@KRBTEST.COM'} at 127.0.0.1:57582
I20260806 15:59:31.476171 32178 tablet_copy_service.cc:433] P 13d4ef44d33d4e888756a75470614006: ending tablet copy session 7bb3bdf4e8a24987b424f27bc1daa914-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7bb3bdf4e8a24987b424f27bc1daa914
I20260806 15:59:31.483279 32640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:31.484917 32756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:31.487313 32759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:31.490451 32640 server_base.cc:1034] running on GCE node
W20260806 15:59:31.494451 32757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:31.494844 32640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:31.495246 32640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:31.496440 32640 hybrid_clock.cc:648] HybridClock initialized: now 1786031971496430 us; error 40 us; skew 500 ppm
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031971, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:31.498406 32640 init.cc:377] Logged in from keytab as oryx/127.25.31.189@KRBTEST.COM (short username oryx)
I20260806 15:59:31.498883 32640 webserver.cc:460] Webserver started at http://127.25.31.189:38111/ using document root <none> and password file <none>
I20260806 15:59:31.499086 32640 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:31.499137 32640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:31.500270 32640 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:59:31.501248 32766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:31.501511 32640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:59:31.501602 32640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/wal
uuid: "7bb3bdf4e8a24987b424f27bc1daa914"
format_stamp: "Formatted at 2026-08-06 15:59:31 on dist-test-slave-tm1g"
server_key: "be6d63cb795a637209f1957c46a25942"
server_key_iv: "d8e88909261371e823e6b0097de852e1"
server_key_version: "encryptionkey@0"
I20260806 15:59:31.501700 32640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:31.512434 32640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:31.514276 32640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:31.519326 32640 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.189:32931
I20260806 15:59:31.519888 32640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-1/data/info.pb
I20260806 15:59:31.520289 352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.189:32931 every 8 connection(s)
I20260806 15:59:31.521454 353 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:59:31.522229 353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: Bootstrap starting.
I20260806 15:59:31.529983 353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:59:31.530270 353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: Bootstrap complete.
I20260806 15:59:31.530742 353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: NON_VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: true } }
I20260806 15:59:31.530875 353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7bb3bdf4e8a24987b424f27bc1daa914, State: Initialized, Role: LEARNER
I20260806 15:59:31.530994 353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: NON_VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: true } }
I20260806 15:59:31.531312 356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: NON_VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: true } } }
I20260806 15:59:31.531468 356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: This master's current role is: LEARNER
I20260806 15:59:31.531488 353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:31.534075 368 catalog_manager.cc:1259] Loaded cluster ID: 4eeb26800d1d43969bcf8b6df3538011
I20260806 15:59:31.534166 368 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: loading cluster ID for follower catalog manager: success
I20260806 15:59:31.535912 368 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: acquiring CA information for follower catalog manager: success
I20260806 15:59:31.536937 368 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 15:59:31.616051 32755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:31.616348 32755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:31.616428 32755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:31.620451 32755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:31.620534 32755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:31.620571 32755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:31.620626 32755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:31.620663 32755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:31.625998 32755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:40393,127.25.31.189:32931,127.25.31.188:32771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.188.keytab
--principal=oryx/127.25.31.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.188:32771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.31.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:31.627125 32755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:31.628162 32755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:31.636690 376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:31.636731 373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:31.636732 374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:31.637046 32755 server_base.cc:1034] running on GCE node
I20260806 15:59:31.637579 32755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:31.638650 32755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:31.639823 32755 hybrid_clock.cc:648] HybridClock initialized: now 1786031971639814 us; error 34 us; skew 500 ppm
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031971, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:31.642400 32755 init.cc:377] Logged in from keytab as oryx/127.25.31.188@KRBTEST.COM (short username oryx)
I20260806 15:59:31.643285 32755 webserver.cc:460] Webserver started at http://127.25.31.188:45569/ using document root <none> and password file <none>
I20260806 15:59:31.643813 32755 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:31.643898 32755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:31.644111 32755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:31.646034 32755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/data/instance:
uuid: "dbe6087f9b104582ba83f141afa0b2ef"
format_stamp: "Formatted at 2026-08-06 15:59:31 on dist-test-slave-tm1g"
server_key: "6e6960fb910ffc633f09b3f580f6dc90"
server_key_iv: "8acdee4a37379baca8c45bcd7d2f522d"
server_key_version: "encryptionkey@0"
I20260806 15:59:31.646643 32755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/wal/instance:
uuid: "dbe6087f9b104582ba83f141afa0b2ef"
format_stamp: "Formatted at 2026-08-06 15:59:31 on dist-test-slave-tm1g"
server_key: "6e6960fb910ffc633f09b3f580f6dc90"
server_key_iv: "8acdee4a37379baca8c45bcd7d2f522d"
server_key_version: "encryptionkey@0"
I20260806 15:59:31.650043 32755 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 15:59:31.652310 383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:31.653429 32755 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 15:59:31.653566 32755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/wal
uuid: "dbe6087f9b104582ba83f141afa0b2ef"
format_stamp: "Formatted at 2026-08-06 15:59:31 on dist-test-slave-tm1g"
server_key: "6e6960fb910ffc633f09b3f580f6dc90"
server_key_iv: "8acdee4a37379baca8c45bcd7d2f522d"
server_key_version: "encryptionkey@0"
I20260806 15:59:31.653666 32755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:31.679894 32755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:31.682683 32755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:31.682859 32755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:31.690042 32755 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.188:32771
I20260806 15:59:31.690347 438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.188:32771 every 8 connection(s)
I20260806 15:59:31.691078 32755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/data/info.pb
I20260806 15:59:31.693683 439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:31.697350 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 32755
I20260806 15:59:31.698557 439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031971, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.188@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.31.188@KRBTEST.COM
I20260806 15:59:31.708513 32356 heartbeater.cc:503] Master 127.25.31.190:40393 was elected leader, sending a full tablet report...
I20260806 15:59:31.710908 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.700349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57598 (local address 127.25.31.190:40393)
0806 15:59:31.700484 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.700489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.700702 (+ 213us) server_negotiation.cc:407] Connection header received
0806 15:59:31.700862 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.700865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.700921 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.700977 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.701939 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.702668 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.703760 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.703995 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.704097 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.706774 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.706777 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.706779 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.706803 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.708339 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.709187 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.709192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.709194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.709285 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.709746 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.709750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.709755 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.709949 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.710033 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.710623 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.710742 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 15:59:31.711854 442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.700181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:57598)
0806 15:59:31.700557 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.700604 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.700732 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.701131 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.701135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.701519 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.701805 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.701818 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.702793 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.702797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.703628 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.703637 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.704141 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.704146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.704289 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.704854 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:31.704870 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:31.706629 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:31.708436 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.708446 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.708455 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.709078 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.709420 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.709425 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.709427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.709613 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.710052 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:31.710060 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:31.710384 (+ 324us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.710577 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.710867 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":150,"threads_started":1}
I20260806 15:59:31.711870 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.699025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39498 (local address 127.25.31.188:32771)
0806 15:59:31.699434 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.699456 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.699489 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:59:31.700107 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.700121 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.700526 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.700871 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.701725 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.702876 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.703740 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.704108 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.705383 (+ 1275us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.708229 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.708234 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.708243 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.708263 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.710208 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.710643 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.710648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.710650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.710729 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.710995 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.710998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.711000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.711249 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.711411 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.711649 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.711753 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":138,"threads_started":1}
I20260806 15:59:31.712970 439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031971, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.188@KRBTEST.COM for oryx/127.25.31.189@KRBTEST.COM
I20260806 15:59:31.720280 442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.713434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:32931 (local address 127.0.0.1:32856)
0806 15:59:31.713574 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.713585 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.713641 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.714012 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.714014 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.714137 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.714222 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.714226 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.715102 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.715105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.715610 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.715617 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.716015 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.716018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.716102 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.716445 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:31.716455 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:31.717588 (+ 1133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:31.719157 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.719160 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.719161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.719349 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.719578 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.719579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.719581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.719653 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.719942 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:31.719944 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:31.720012 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.720088 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.720179 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 15:59:31.720350 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.713546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32856 (local address 127.25.31.189:32931)
0806 15:59:31.713793 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.713798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.713812 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:59:31.713857 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.713860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.713908 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.713974 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.714353 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.715014 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.715699 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.715908 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.715981 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.717713 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.717716 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.717718 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.717739 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.719064 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.719434 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.719436 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.719437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.719505 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.719722 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.719724 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.719725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.719872 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.719943 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.720137 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.720226 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":81,"threads_started":1}
I20260806 15:59:31.721055 439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031971, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.188@KRBTEST.COM for oryx/127.25.31.188@KRBTEST.COM
I20260806 15:59:31.728482 442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.721487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:32771 (local address 127.0.0.1:39504)
0806 15:59:31.721589 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.721602 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.721673 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.721879 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.721882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.722031 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.722088 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.722092 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.722876 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.722879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.723346 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.723352 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.723796 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.723800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.723881 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.724299 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:31.724311 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:31.725378 (+ 1067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:31.727146 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.727149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.727151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.727329 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.727590 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.727593 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.727595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.727673 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.728014 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:31.728017 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:31.728093 (+ 76us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.728196 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.728323 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 15:59:31.728508 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.721528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39504 (local address 127.25.31.188:32771)
0806 15:59:31.721628 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.721634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.721649 (+ 15us) server_negotiation.cc:407] Connection header received
0806 15:59:31.721750 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.721752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.721782 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.721863 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.722189 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.722752 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.723452 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.723696 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.723797 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.725515 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.725518 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.725520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.725542 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.727038 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.727417 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.727422 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.727424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.727497 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.727753 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.727756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.727759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.727925 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.728024 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.728239 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.728384 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":40}
I20260806 15:59:31.731539 439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: Bootstrap starting.
I20260806 15:59:31.733916 439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:31.734689 439 log.cc:824] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:31.736410 439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: No bootstrap required, opened a new log
I20260806 15:59:31.739351 439 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } }
I20260806 15:59:31.739602 439 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:31.739662 439 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe6087f9b104582ba83f141afa0b2ef, State: Initialized, Role: FOLLOWER
I20260806 15:59:31.740101 439 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } }
I20260806 15:59:31.740610 445 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } } }
I20260806 15:59:31.740768 445 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:59:31.741588 439 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [sys.catalog]: configured and running, proceeding with master startup.
W20260806 15:59:31.746464 456 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:59:31.746594 456 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:59:31.750666 32755 master_runner.cc:428] Detected that this master dbe6087f9b104582ba83f141afa0b2ef is joining an existing cluster
I20260806 15:59:31.750726 32755 master_runner.cc:432] Initiating AddMaster RPC to add 127.25.31.188:32771
I20260806 15:59:31.761950 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.753666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39512 (local address 127.25.31.188:32771)
0806 15:59:31.753759 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.753763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.754432 (+ 669us) server_negotiation.cc:407] Connection header received
0806 15:59:31.754571 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.754574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.754607 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.754650 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.755204 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.755871 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.756973 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.757346 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.757748 (+ 402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.759176 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.759179 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.759182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.759203 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.760451 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.760840 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.760847 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.760849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.760923 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.761174 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.761180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.761182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.761345 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.761439 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.761682 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.761820 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":39}
I20260806 15:59:31.761962 464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.753845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:32771 (local address 127.0.0.1:39512)
0806 15:59:31.754356 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.754369 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.754478 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.754748 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.754750 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.754938 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.755100 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.755106 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.756027 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.756030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.756865 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.756873 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.757884 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.757887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.758254 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.758701 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:31.758715 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:31.759052 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:31.760560 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.760566 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.760568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.760753 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.761011 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.761015 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.761017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.761095 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.761440 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:31.761443 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:31.761527 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.761638 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.761770 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":1,"thread_start_us":79,"threads_started":1}
I20260806 15:59:31.764977 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 32220
I20260806 15:59:31.773104 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:41197
--local_ip_for_outbound_sockets=127.25.31.129
--tserver_master_addrs=127.25.31.190:40393
--webserver_port=39047
--webserver_interface=127.25.31.129
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.129.keytab
--principal=oryx/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.31.190:40393,127.25.31.189:32931
--tserver_master_addrs=127.25.31.190:40393,127.25.31.189:32931,127.25.31.188:32771 with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:59:31.778460 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.753010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:32931 (local address 127.0.0.1:32860)
0806 15:59:31.753446 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.753460 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.753597 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.753980 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.753983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.754275 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.754449 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.754457 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.755499 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.755514 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.756366 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.756374 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.757128 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.757131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.757234 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.757749 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:31.757763 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:31.774648 (+ 16885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:31.776680 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.776684 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.776687 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.776882 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.777148 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.777152 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.777154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.777239 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.777907 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:31.777910 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:31.777998 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.778095 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.778236 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":2,"thread_start_us":98,"threads_started":1}
I20260806 15:59:31.778650 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.753164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57612 (local address 127.25.31.190:40393)
0806 15:59:31.753290 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.753295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.753660 (+ 365us) server_negotiation.cc:407] Connection header received
0806 15:59:31.753828 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.753831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.753896 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.753949 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.754699 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.755539 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.756833 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.757266 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.757365 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.775107 (+ 17742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.775111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.775114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.775149 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.776869 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.777334 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.777337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.777338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.777411 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.777730 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.777733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.777735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.777947 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.778025 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.778375 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.778497 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 15:59:31.778998 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.753166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32860 (local address 127.25.31.189:32931)
0806 15:59:31.753288 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.753293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.753526 (+ 233us) server_negotiation.cc:407] Connection header received
0806 15:59:31.753711 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.753715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.753764 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.753820 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.754586 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.755387 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.756486 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.756870 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.756972 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.774829 (+ 17857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.774836 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.774841 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.774875 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.776568 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.776977 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.776981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.776983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.777061 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.777531 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.777534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.777536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.777776 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.777864 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.778659 (+ 795us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.778800 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 15:59:31.779856 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.752999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:57612)
0806 15:59:31.753573 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.753588 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.753723 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.754112 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.754115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.754404 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.754544 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.754552 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.755710 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.755713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.756698 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.756709 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.757599 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.757602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.757777 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.758222 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:31.758237 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:31.774954 (+ 16717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:31.776997 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.777000 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.777002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.777236 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.777507 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.777510 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.777514 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.777616 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.778098 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:31.778102 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:31.778174 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.779532 (+ 1358us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.779691 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":46,"thread_start_us":91,"threads_started":1}
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031970, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.190@KRBTEST.COM for oryx/127.25.31.188@KRBTEST.COM
I20260806 15:59:31.790625 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.783283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39528 (local address 127.25.31.188:32771)
0806 15:59:31.783383 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.783389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.783439 (+ 50us) server_negotiation.cc:407] Connection header received
0806 15:59:31.783586 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.783589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.783637 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.783714 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.784119 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.784608 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.785365 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.785601 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.785699 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.787631 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.787635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.787637 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.787659 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.789087 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.789483 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.789487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.789489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.789567 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.789826 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.789831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.789833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.789991 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.790127 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.790390 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.790485 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 15:59:31.790642 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.783229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:32771 (local address 127.0.0.1:39528)
0806 15:59:31.783383 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.783397 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.783479 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.783729 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.783732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.783905 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.783983 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.783988 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.784707 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.784710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.785252 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.785260 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.785719 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.785723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.785826 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.786249 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:31.786262 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:31.787499 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:31.789193 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.789197 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.789198 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.789386 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.789659 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.789662 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.789664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.789744 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.790118 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:31.790121 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:31.790219 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.790350 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.790487 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 15:59:31.791741 32156 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.25.31.188:32771: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef"
member_type: NON_VOTER
last_known_addr {
host: "127.25.31.188"
port: 32771
}
attrs {
promote: true
}
}
dest_uuid: "13d4ef44d33d4e888756a75470614006"
cas_config_opid_index: 5
I20260806 15:59:31.792294 32156 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: NON_VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: true } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } }
I20260806 15:59:31.793365 32741 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER dbe6087f9b104582ba83f141afa0b2ef (127.25.31.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: NON_VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: true } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } } }
I20260806 15:59:31.793964 411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:31.794998 32741 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.25.31.188:32771
I20260806 15:59:31.795161 32743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: NON_VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: true } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } } }
I20260806 15:59:31.795292 32743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:31.795485 411 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 1 FOLLOWER]: Refusing update from remote peer 13d4ef44d33d4e888756a75470614006: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 15:59:31.795686 32755 master.cc:537] Master@127.25.31.188:32771 shutting down...
I20260806 15:59:31.796649 32743 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 15:59:31.799206 32139 consensus_peers.cc:487] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 -> Peer dbe6087f9b104582ba83f141afa0b2ef (127.25.31.188:32771): Couldn't send request to peer dbe6087f9b104582ba83f141afa0b2ef. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 15:59:31.799705 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.793600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:32931 (local address 127.0.0.1:32876)
0806 15:59:31.793922 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.793938 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.794015 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.794607 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.794610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.794626 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:31.794707 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.794712 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.796855 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.796858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.798045 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.798055 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.799271 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.799274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.799369 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.799426 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.799519 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.799587 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256}
I20260806 15:59:31.799706 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.793812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32876 (local address 127.25.31.189:32931)
0806 15:59:31.794172 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.794179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.794195 (+ 16us) server_negotiation.cc:407] Connection header received
0806 15:59:31.794239 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.794242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.794380 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.794473 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:31.794822 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.796723 (+ 1901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.798174 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.798957 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.799051 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.799211 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.799535 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.799602 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278}
I20260806 15:59:31.800496 326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 LEARNER]: Refusing update from remote peer 13d4ef44d33d4e888756a75470614006: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 15:59:31.800879 32755 raft_consensus.cc:2229] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:31.801064 32743 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: NON_VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 15:59:31.801152 32755 raft_consensus.cc:2258] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:31.801254 32755 tablet_replica.cc:330] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: stopping tablet replica
I20260806 15:59:31.803272 326 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER dbe6087f9b104582ba83f141afa0b2ef (127.25.31.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: NON_VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: true } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } } }
I20260806 15:59:31.805034 468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: NON_VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: true } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } } }
I20260806 15:59:31.805161 468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: This master's current role is: LEARNER
I20260806 15:59:31.805477 356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13d4ef44d33d4e888756a75470614006. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: NON_VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: true } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } } }
I20260806 15:59:31.805546 356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: This master's current role is: LEARNER
I20260806 15:59:31.807871 32741 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006: attempting to promote NON_VOTER 7bb3bdf4e8a24987b424f27bc1daa914 to VOTER
I20260806 15:59:31.807879 32755 master.cc:552] Master@127.25.31.188:32771 shutdown complete.
I20260806 15:59:31.807926 32755 master_runner.cc:305] Clearing existing system tablet
I20260806 15:59:31.808141 32755 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:31.808212 32755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:31.808277 32741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } }
I20260806 15:59:31.809469 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.809118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:32771 (local address 127.0.0.1:39534)
0806 15:59:31.809324 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.809400 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:32771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260806 15:59:31.809530 326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 LEARNER]: Refusing update from remote peer 13d4ef44d33d4e888756a75470614006: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 15:59:31.809872 32755 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:59:31.810087 32743 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 15:59:31.810483 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.809893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:32771 (local address 127.0.0.1:39550)
0806 15:59:31.809983 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.810032 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:32771: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 15:59:31.810557 32139 consensus_peers.cc:487] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 -> Peer dbe6087f9b104582ba83f141afa0b2ef (127.25.31.188:32771): Couldn't send request to peer dbe6087f9b104582ba83f141afa0b2ef. Status: Network error: Client connection negotiation failed: client connection to 127.25.31.188:32771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 15:59:31.811208 470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:31.811471 32755 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:31.811764 32755 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 15:59:31.811872 32755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/wal
uuid: "dbe6087f9b104582ba83f141afa0b2ef"
format_stamp: "Formatted at 2026-08-06 15:59:31 on dist-test-slave-tm1g"
server_key: "6e6960fb910ffc633f09b3f580f6dc90"
server_key_iv: "8acdee4a37379baca8c45bcd7d2f522d"
server_key_version: "encryptionkey@0"
I20260806 15:59:31.811784 32742 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7bb3bdf4e8a24987b424f27bc1daa914 (127.25.31.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } } }
I20260806 15:59:31.812199 326 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7bb3bdf4e8a24987b424f27bc1daa914 (127.25.31.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } } }
I20260806 15:59:31.813247 32742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } } }
I20260806 15:59:31.813366 32742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:31.813503 32742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } } }
I20260806 15:59:31.813585 32742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:31.813580 356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } } }
I20260806 15:59:31.813687 356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:59:31.814528 32755 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:31.816300 32755 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 15:59:31.816402 32755 log.cc:1196] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 15:59:31.816638 32755 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: Deleting consensus metadata
I20260806 15:59:31.816818 32755 master_runner.cc:315] Copying system tablet from 127.25.31.190:40393
I20260806 15:59:31.817857 32755 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: tablet copy: Beginning tablet copy session from remote peer at address 127.25.31.190:40393
I20260806 15:59:31.829654 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.822689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.25.31.190:40393)
0806 15:59:31.822806 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.822810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.823022 (+ 212us) server_negotiation.cc:407] Connection header received
0806 15:59:31.823156 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.823159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.823206 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.823285 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.823725 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.824256 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.825102 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.825335 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.825448 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.826621 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:31.826625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:31.826628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:31.826651 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:31.828062 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.828480 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.828484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.828486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.828565 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:31.828836 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:31.828840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:31.828848 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:31.829016 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:31.829122 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:31.829397 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 15:59:31.829502 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 15:59:31.829669 476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.822661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:57628)
0806 15:59:31.822935 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.822947 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.823068 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.823302 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.823305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.823512 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.823612 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.823618 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.824372 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.824376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.824987 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.824995 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:31.825461 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:31.825465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:31.825561 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:31.826103 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:31.826114 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:31.826491 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:31.828172 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.828176 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.828179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.828385 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.828662 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:31.828664 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:31.828666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:31.828749 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:31.829112 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:31.829115 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:31.829235 (+ 120us) client_negotiation.cc:769] Sending connection context
0806 15:59:31.829340 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 15:59:31.829488 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":96,"threads_started":1}
I20260806 15:59:31.830426 32178 tablet_copy_service.cc:140] P 13d4ef44d33d4e888756a75470614006: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer dbe6087f9b104582ba83f141afa0b2ef ({username='oryx', principal='oryx/127.25.31.188@KRBTEST.COM'} at 127.0.0.1:57628)
I20260806 15:59:31.830524 32178 tablet_copy_service.cc:161] P 13d4ef44d33d4e888756a75470614006: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer dbe6087f9b104582ba83f141afa0b2ef at {username='oryx', principal='oryx/127.25.31.188@KRBTEST.COM'} at 127.0.0.1:57628: session id = dbe6087f9b104582ba83f141afa0b2ef-00000000000000000000000000000000
I20260806 15:59:31.831305 32178 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 15:59:31.832449 32755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 15:59:31.834468 32755 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: tablet copy: Starting download of 0 data blocks...
I20260806 15:59:31.834652 32755 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: tablet copy: Starting download of 1 WAL segments...
I20260806 15:59:31.835913 32755 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 15:59:31.837293 32178 tablet_copy_service.cc:342] P 13d4ef44d33d4e888756a75470614006: Request end of tablet copy session dbe6087f9b104582ba83f141afa0b2ef-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.25.31.188@KRBTEST.COM'} at 127.0.0.1:57628
I20260806 15:59:31.837380 32178 tablet_copy_service.cc:433] P 13d4ef44d33d4e888756a75470614006: ending tablet copy session dbe6087f9b104582ba83f141afa0b2ef-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer dbe6087f9b104582ba83f141afa0b2ef
I20260806 15:59:31.843230 32755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:31.844794 477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:31.844861 478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:31.845091 481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:31.845371 32755 server_base.cc:1034] running on GCE node
I20260806 15:59:31.845530 32755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:31.845819 32755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:31.846951 32755 hybrid_clock.cc:648] HybridClock initialized: now 1786031971846942 us; error 24 us; skew 500 ppm
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031971, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:31.848311 32755 init.cc:377] Logged in from keytab as oryx/127.25.31.188@KRBTEST.COM (short username oryx)
I20260806 15:59:31.848747 32755 webserver.cc:460] Webserver started at http://127.25.31.188:38809/ using document root <none> and password file <none>
I20260806 15:59:31.848948 32755 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:31.848999 32755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:31.850100 32755 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 15:59:31.851091 488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:31.851312 32755 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:59:31.851454 32755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/wal
uuid: "dbe6087f9b104582ba83f141afa0b2ef"
format_stamp: "Formatted at 2026-08-06 15:59:31 on dist-test-slave-tm1g"
server_key: "6e6960fb910ffc633f09b3f580f6dc90"
server_key_iv: "8acdee4a37379baca8c45bcd7d2f522d"
server_key_version: "encryptionkey@0"
I20260806 15:59:31.851549 32755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:31.860599 32755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:31.862481 32755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:31.867455 32755 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.188:32771
I20260806 15:59:31.867928 540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.188:32771 every 8 connection(s)
I20260806 15:59:31.868230 32755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/master-2/data/info.pb
I20260806 15:59:31.870002 541 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:59:31.871083 541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: Bootstrap starting.
I20260806 15:59:31.881378 541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:59:31.881654 541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: Bootstrap complete.
I20260806 15:59:31.882179 541 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } }
I20260806 15:59:31.882339 541 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dbe6087f9b104582ba83f141afa0b2ef, State: Initialized, Role: LEARNER
I20260806 15:59:31.882494 541 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } }
I20260806 15:59:31.882943 541 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:31.882890 544 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } } }
I20260806 15:59:31.883126 544 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [sys.catalog]: This master's current role is: LEARNER
I20260806 15:59:31.885686 556 catalog_manager.cc:1259] Loaded cluster ID: 4eeb26800d1d43969bcf8b6df3538011
I20260806 15:59:31.885776 556 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: loading cluster ID for follower catalog manager: success
I20260806 15:59:31.888032 556 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: acquiring CA information for follower catalog manager: success
I20260806 15:59:31.889525 556 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 15:59:31.925127 465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:31.925369 465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:31.925415 465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:31.929384 465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:31.929443 465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:31.929531 465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:31.930188 32497 heartbeater.cc:503] Master 127.25.31.190:40393 was elected leader, sending a full tablet report...
I20260806 15:59:31.935099 465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.129.keytab
--principal=oryx/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:41197
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=39047
--tserver_master_addrs=127.25.31.190:40393,127.25.31.189:32931,127.25.31.188:32771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:31.936094 465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:31.937075 465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:31.946223 562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:31.946231 561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:31.946240 564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:31.946333 465 server_base.cc:1034] running on GCE node
I20260806 15:59:31.946843 465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:31.947404 465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:31.948562 465 hybrid_clock.cc:648] HybridClock initialized: now 1786031971948548 us; error 26 us; skew 500 ppm
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031971, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:31.951249 465 init.cc:377] Logged in from keytab as oryx/127.25.31.129@KRBTEST.COM (short username oryx)
I20260806 15:59:31.952142 465 webserver.cc:460] Webserver started at http://127.25.31.129:39047/ using document root <none> and password file <none>
I20260806 15:59:31.952775 465 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:31.952857 465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:31.956807 465 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 15:59:31.958864 571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:31.960006 465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:31.960121 465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b"
format_stamp: "Formatted at 2026-08-06 15:59:30 on dist-test-slave-tm1g"
server_key: "d81d8bd1a28da9c2765ab227113e47a1"
server_key_iv: "a44a1132fd98f668f5960b75c4b42806"
server_key_version: "encryptionkey@0"
I20260806 15:59:31.960628 465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:31.976003 465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:31.979136 465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:31.979336 465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:31.979939 465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:31.980767 465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:31.980814 465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:31.980880 465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:31.980911 465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:31.989665 465 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:41197
I20260806 15:59:31.989732 689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:41197 every 8 connection(s)
I20260806 15:59:31.990694 465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:31.991727 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 465
I20260806 15:59:31.991927 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 32361
I20260806 15:59:32.001652 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:43327
--local_ip_for_outbound_sockets=127.25.31.130
--tserver_master_addrs=127.25.31.190:40393
--webserver_port=39591
--webserver_interface=127.25.31.130
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.130.keytab
--principal=oryx/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.31.190:40393,127.25.31.189:32931
--tserver_master_addrs=127.25.31.190:40393,127.25.31.189:32931,127.25.31.188:32771 with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031971, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.189@KRBTEST.COM
Aug 06 15:59:31 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031971, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.188@KRBTEST.COM
Aug 06 15:59:32 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031971, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
I20260806 15:59:32.010625 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.995413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:33759 (local address 127.25.31.189:32931)
0806 15:59:31.995548 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.995553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.998341 (+ 2788us) server_negotiation.cc:407] Connection header received
0806 15:59:31.998559 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.998562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.998614 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.998685 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.999488 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.000326 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.002436 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.002848 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.002944 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.006087 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.006091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.006094 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.006124 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.007937 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.008632 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.008639 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.008644 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.008739 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.009186 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.009189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.009191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.009451 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.009532 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.010254 (+ 722us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.010436 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 15:59:32.011445 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.993246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:58051 (local address 127.25.31.190:40393)
0806 15:59:31.993381 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.993386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.996411 (+ 3025us) server_negotiation.cc:407] Connection header received
0806 15:59:31.997391 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.997395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.997458 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.997546 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.999499 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.000383 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.002424 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.002782 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.002875 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.007446 (+ 4571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.007450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.007453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.007486 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.009590 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.010055 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.010059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.010061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.010155 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.010499 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.010502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.010504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.010720 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.010832 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.011185 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.011304 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 15:59:32.012475 696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.995028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:32931 (local address 127.25.31.129:33759)
0806 15:59:31.998237 (+ 3209us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.998250 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.998415 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.998796 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.998798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.998979 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.999325 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.999337 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.000506 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.000509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.002250 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.002265 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.003132 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.003137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.003301 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.004053 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:32.004072 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:32.005877 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:32.008097 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.008105 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.008120 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.008502 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.008862 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.008866 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.008871 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.009061 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.009625 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:32.009635 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:32.009963 (+ 328us) client_negotiation.cc:769] Sending connection context
0806 15:59:32.010901 (+ 938us) client_negotiation.cc:240] Negotiation successful
0806 15:59:32.011266 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3145,"mutex_wait_us":423,"spinlock_wait_cycles":8064,"thread_start_us":109,"threads_started":1}
I20260806 15:59:32.012741 694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.992869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.129:58051)
0806 15:59:31.996248 (+ 3379us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.996308 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.997183 (+ 875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.997765 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.997778 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.998426 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.999310 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.999334 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.000543 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.000549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.002253 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.002264 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.003093 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.003097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.003291 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.004012 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:32.004051 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:32.007271 (+ 3220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:32.009695 (+ 2424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.009697 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.009699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.009937 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.010264 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.010267 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.010268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.010386 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.010898 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:32.010901 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:32.010989 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 15:59:32.011654 (+ 665us) client_negotiation.cc:240] Negotiation successful
0806 15:59:32.011796 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3201,"mutex_wait_us":285,"spinlock_wait_cycles":42112,"thread_start_us":126,"threads_started":1}
I20260806 15:59:32.013036 697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.995253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:51883 (local address 127.25.31.188:32771)
0806 15:59:31.995519 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:31.995533 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:31.997742 (+ 2209us) server_negotiation.cc:407] Connection header received
0806 15:59:31.997786 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:31.997788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:31.997840 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:31.997931 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:31.999489 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.000557 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.003099 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.003463 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.003565 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.007373 (+ 3808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.007380 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.007386 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.007424 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.009673 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.010257 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.010264 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.010269 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.010506 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.011107 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.011113 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.011117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.011394 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.011503 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.012616 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.012780 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":94,"threads_started":1}
I20260806 15:59:32.012490 695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:31.993902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:32771 (local address 127.25.31.129:51883)
0806 15:59:31.997574 (+ 3672us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:31.997588 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:31.997670 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:31.997948 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:31.997951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:31.998425 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:31.999331 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:31.999337 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.000686 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.000689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.002919 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.002929 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.003738 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.003741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.003812 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.004191 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:32.004206 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:32.006928 (+ 2722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:32.009819 (+ 2891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.009824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.009829 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.010124 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.010637 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.010643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.010648 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.010745 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.011727 (+ 982us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:32.011732 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:32.011864 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 15:59:32.011997 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 15:59:32.012215 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3605,"mutex_wait_us":1284,"thread_start_us":125,"threads_started":1}
I20260806 15:59:32.014035 691 heartbeater.cc:347] Connected to a master server at 127.25.31.190:40393
I20260806 15:59:32.014387 692 heartbeater.cc:347] Connected to a master server at 127.25.31.189:32931
I20260806 15:59:32.014483 692 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:32.015137 692 heartbeater.cc:511] Master 127.25.31.189:32931 requested a full tablet report, sending...
I20260806 15:59:32.015959 691 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:32.016207 691 heartbeater.cc:511] Master 127.25.31.190:40393 requested a full tablet report, sending...
I20260806 15:59:32.016853 690 heartbeater.cc:347] Connected to a master server at 127.25.31.188:32771
I20260806 15:59:32.016942 690 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:32.017092 690 heartbeater.cc:511] Master 127.25.31.188:32771 requested a full tablet report, sending...
I20260806 15:59:32.017149 32156 ts_manager.cc:194] Re-registered known tserver with Master: dfad7b8c48ec447ca43f3dc53e5cf34b (127.25.31.129:41197)
I20260806 15:59:32.017593 315 ts_manager.cc:194] Registered new tserver with Master: dfad7b8c48ec447ca43f3dc53e5cf34b (127.25.31.129:41197)
I20260806 15:59:32.017851 32156 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:58051
I20260806 15:59:32.018088 501 ts_manager.cc:194] Registered new tserver with Master: dfad7b8c48ec447ca43f3dc53e5cf34b (127.25.31.129:41197)
W20260806 15:59:32.119506 698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:32.119740 698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:32.119782 698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:32.123739 698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:32.123791 698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:32.123847 698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:32.129025 698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.130.keytab
--principal=oryx/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:43327
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=39591
--tserver_master_addrs=127.25.31.190:40393,127.25.31.189:32931,127.25.31.188:32771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:32.130029 698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:32.131016 698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:32.140327 703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:32.140367 706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:32.140362 704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:32.140508 698 server_base.cc:1034] running on GCE node
I20260806 15:59:32.140941 698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:32.141532 698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:32.142730 698 hybrid_clock.cc:648] HybridClock initialized: now 1786031972142704 us; error 33 us; skew 500 ppm
Aug 06 15:59:32 dist-test-slave-tm1g krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031972, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:32.145385 698 init.cc:377] Logged in from keytab as oryx/127.25.31.130@KRBTEST.COM (short username oryx)
I20260806 15:59:32.146189 698 webserver.cc:460] Webserver started at http://127.25.31.130:39591/ using document root <none> and password file <none>
I20260806 15:59:32.146772 698 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:32.146875 698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:32.150601 698 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 15:59:32.152626 713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:32.153831 698 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:32.153973 698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "1d0f242a13004c89911dcd52f996080d"
format_stamp: "Formatted at 2026-08-06 15:59:30 on dist-test-slave-tm1g"
server_key: "4ecf8bddbafda8b15d616c20171d4149"
server_key_iv: "fe81bf67dbe3671aa2a165e9d312ec90"
server_key_version: "encryptionkey@0"
I20260806 15:59:32.154542 698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:32.154816 32635 heartbeater.cc:503] Master 127.25.31.190:40393 was elected leader, sending a full tablet report...
I20260806 15:59:32.168553 698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:32.171355 698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:32.171546 698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:32.172119 698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:32.172952 698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:32.173019 698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:32.173089 698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:32.173130 698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:32.182173 698 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:43327
I20260806 15:59:32.182204 830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:43327 every 8 connection(s)
I20260806 15:59:32.183192 698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:32.190900 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 698
I20260806 15:59:32.191102 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 32502
Aug 06 15:59:32 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031972, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.188@KRBTEST.COM
Aug 06 15:59:32 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031972, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
Aug 06 15:59:32 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031972, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.189@KRBTEST.COM
I20260806 15:59:32.199805 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.186598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:36615 (local address 127.25.31.189:32931)
0806 15:59:32.186716 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.186721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.186863 (+ 142us) server_negotiation.cc:407] Connection header received
0806 15:59:32.187321 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.187324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.187380 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.187442 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:32.188751 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.189555 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.190935 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.191309 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.191400 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.195749 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.195753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.195761 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.195784 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.197526 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.198028 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.198034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.198036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.198123 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.198468 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.198471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.198473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.198696 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.198768 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.199519 (+ 751us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.199635 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 15:59:32.200711 697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.184985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:54737 (local address 127.25.31.188:32771)
0806 15:59:32.185117 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.185121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.186278 (+ 1157us) server_negotiation.cc:407] Connection header received
0806 15:59:32.187317 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.187321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.187385 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.187479 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:32.188731 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.189272 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.190496 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.190804 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.190918 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.195113 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.195117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.195120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.195149 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.197171 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.197866 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.197872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.197877 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.197968 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.198413 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.198419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.198424 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.198687 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.198774 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.200313 (+ 1539us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.200478 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 15:59:32.201097 838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.186466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:32931 (local address 127.25.31.130:36615)
0806 15:59:32.186783 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:32.186793 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:32.187126 (+ 333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:32.187644 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:32.187652 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:32.188140 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:32.188589 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.188603 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.189688 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.189690 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.190789 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.190801 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.191650 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.191657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.191793 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.192248 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:32.192268 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:32.195576 (+ 3308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:32.197658 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.197661 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.197663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.197915 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.198236 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.198238 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.198240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.198362 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.198847 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:32.198853 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:32.199240 (+ 387us) client_negotiation.cc:769] Sending connection context
0806 15:59:32.200224 (+ 984us) client_negotiation.cc:240] Negotiation successful
0806 15:59:32.200358 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":236,"spinlock_wait_cycles":47232,"thread_start_us":85,"threads_started":1}
I20260806 15:59:32.201251 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.184983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:57033 (local address 127.25.31.190:40393)
0806 15:59:32.185116 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.185121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.186275 (+ 1154us) server_negotiation.cc:407] Connection header received
0806 15:59:32.187405 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.187411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.187466 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.187534 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:32.188745 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.189563 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.190927 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.191235 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.191332 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.195079 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.195085 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.195091 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.195124 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.197780 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.198235 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.198239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.198241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.198350 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.198655 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.198658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.198659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.198881 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.198974 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.200966 (+ 1992us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.201086 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 15:59:32.201287 836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.185418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:32771 (local address 127.25.31.130:54737)
0806 15:59:32.186126 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:32.186180 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:32.187116 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:32.187644 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:32.187650 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:32.188101 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:32.188587 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.188599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.189420 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.189440 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.190368 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.190383 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.191071 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.191075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.191252 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.192292 (+ 1040us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:32.192307 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:32.194889 (+ 2582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:32.197347 (+ 2458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.197356 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.197374 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.197745 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.198077 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.198082 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.198087 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.198265 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.198844 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:32.198853 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:32.199242 (+ 389us) client_negotiation.cc:769] Sending connection context
0806 15:59:32.200858 (+ 1616us) client_negotiation.cc:240] Negotiation successful
0806 15:59:32.201072 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":52,"spinlock_wait_cycles":20864,"thread_start_us":143,"threads_started":1}
I20260806 15:59:32.201577 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:37269
--local_ip_for_outbound_sockets=127.25.31.131
--tserver_master_addrs=127.25.31.190:40393
--webserver_port=40117
--webserver_interface=127.25.31.131
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.131.keytab
--principal=oryx/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.31.190:40393,127.25.31.189:32931
--tserver_master_addrs=127.25.31.190:40393,127.25.31.189:32931,127.25.31.188:32771 with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:59:32.201475 837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.185418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.130:57033)
0806 15:59:32.186125 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:32.186180 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:32.187126 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:32.187644 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:32.187652 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:32.188134 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:32.188590 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.188603 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.189685 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.189688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.190797 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.190805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.191464 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.191469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.191588 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.192121 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:32.192146 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:32.194892 (+ 2746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:32.197898 (+ 3006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.197900 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.197902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.198124 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.198467 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.198470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.198472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.198555 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.199016 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:32.199019 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:32.199240 (+ 221us) client_negotiation.cc:769] Sending connection context
0806 15:59:32.199449 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 15:59:32.199759 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"mutex_wait_us":184,"spinlock_wait_cycles":49536,"thread_start_us":131,"threads_started":1}
I20260806 15:59:32.202802 834 heartbeater.cc:347] Connected to a master server at 127.25.31.189:32931
I20260806 15:59:32.203055 834 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:32.203173 832 heartbeater.cc:347] Connected to a master server at 127.25.31.188:32771
I20260806 15:59:32.203250 832 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:32.203482 833 heartbeater.cc:347] Connected to a master server at 127.25.31.190:40393
I20260806 15:59:32.203552 833 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:32.203644 834 heartbeater.cc:511] Master 127.25.31.189:32931 requested a full tablet report, sending...
I20260806 15:59:32.203648 832 heartbeater.cc:511] Master 127.25.31.188:32771 requested a full tablet report, sending...
I20260806 15:59:32.203683 833 heartbeater.cc:511] Master 127.25.31.190:40393 requested a full tablet report, sending...
I20260806 15:59:32.204721 32156 ts_manager.cc:194] Re-registered known tserver with Master: 1d0f242a13004c89911dcd52f996080d (127.25.31.130:43327)
I20260806 15:59:32.204793 315 ts_manager.cc:194] Registered new tserver with Master: 1d0f242a13004c89911dcd52f996080d (127.25.31.130:43327)
I20260806 15:59:32.205505 32156 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:57033
I20260806 15:59:32.206030 501 ts_manager.cc:194] Registered new tserver with Master: 1d0f242a13004c89911dcd52f996080d (127.25.31.130:43327)
W20260806 15:59:32.314956 839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:32.315238 839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:32.315321 839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:32.319401 839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:32.319486 839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:32.319574 839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:32.324854 839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.131.keytab
--principal=oryx/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:37269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=40117
--tserver_master_addrs=127.25.31.190:40393,127.25.31.189:32931,127.25.31.188:32771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:32.325948 839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:32.327042 839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:32.336925 845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:32.336897 844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:32.336897 847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:32.337352 839 server_base.cc:1034] running on GCE node
I20260806 15:59:32.337713 839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:32.338352 839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:32.339529 839 hybrid_clock.cc:648] HybridClock initialized: now 1786031972339513 us; error 26 us; skew 500 ppm
Aug 06 15:59:32 dist-test-slave-tm1g krb5kdc[32110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031972, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:32.342255 839 init.cc:377] Logged in from keytab as oryx/127.25.31.131@KRBTEST.COM (short username oryx)
I20260806 15:59:32.343135 839 webserver.cc:460] Webserver started at http://127.25.31.131:40117/ using document root <none> and password file <none>
I20260806 15:59:32.343696 839 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:32.343806 839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:32.347661 839 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 15:59:32.349659 854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:32.351141 839 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:32.351274 839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "8b17d836eb794e359e1e0ceecb09f970"
format_stamp: "Formatted at 2026-08-06 15:59:31 on dist-test-slave-tm1g"
server_key: "fc44bd11e85f1b3c996c93a61afc0910"
server_key_iv: "5292cb90a84f298e3e49c5c7facf10ef"
server_key_version: "encryptionkey@0"
I20260806 15:59:32.351874 839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:32.363487 839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:32.366753 839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:32.366954 839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:32.367564 839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:32.368422 839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:32.368489 839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:32.368558 839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:32.368605 839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:32.378300 839 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:37269
I20260806 15:59:32.378355 974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:37269 every 8 connection(s)
I20260806 15:59:32.379369 839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 15:59:32.388657 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 839
I20260806 15:59:32.390446 984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.385188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:32771 (local address 127.0.0.1:39560)
0806 15:59:32.385602 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:32.385618 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:32.385709 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:32.386271 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:32.386274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:32.386292 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:32.386428 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.386435 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.387424 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.387427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.388595 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.388607 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.389910 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.389916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.390036 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.390066 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 15:59:32.390148 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 15:59:32.390244 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":64,"threads_started":1}
Aug 06 15:59:32 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031972, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.188@KRBTEST.COM
Aug 06 15:59:32 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031972, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.189@KRBTEST.COM
I20260806 15:59:32.397251 985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.385519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39560 (local address 127.25.31.188:32771)
0806 15:59:32.385942 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.385946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.385957 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:59:32.385993 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.385995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.386069 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.386170 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:32.386547 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.387288 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.388865 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.389554 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.389674 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.389840 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.390185 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.390269 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":76,"threads_started":1}
I20260806 15:59:32.397902 515 raft_consensus.cc:1203] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 15:59:32.399475 697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.381167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:58079 (local address 127.25.31.188:32771)
0806 15:59:32.381308 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.381312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.382421 (+ 1109us) server_negotiation.cc:407] Connection header received
0806 15:59:32.383825 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.383829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.383888 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.383965 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:32.385841 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.386600 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.388630 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.388977 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.389072 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.393668 (+ 4596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.393671 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.393675 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.393701 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.395481 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.396149 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.396153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.396156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.396252 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.396622 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.396627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.396630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.396866 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.396977 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.399138 (+ 2161us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.399285 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 15:59:32.400018 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.382968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:55465 (local address 127.25.31.189:32931)
0806 15:59:32.383134 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.383137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.383248 (+ 111us) server_negotiation.cc:407] Connection header received
0806 15:59:32.383903 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.383907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.383963 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.384026 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:32.385739 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.386606 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.388631 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.388977 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.389068 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.396450 (+ 7382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.396455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.396459 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.396491 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.398195 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.398722 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.398725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.398727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.398814 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.399115 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.399119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.399121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.399319 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.399396 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.399768 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.399881 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 15:59:32.400286 996 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13d4ef44d33d4e888756a75470614006. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } } }
I20260806 15:59:32.400058 980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.381587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:32771 (local address 127.25.31.131:58079)
0806 15:59:32.382245 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:32.382323 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:32.383622 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:32.384186 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:32.384199 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:32.384710 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:32.385691 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.385701 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.386747 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.386750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.388492 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.388498 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.389241 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.389244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.389576 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.390505 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:32.390526 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:32.393481 (+ 2955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:32.395608 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.395614 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.395630 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.396029 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.396338 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.396342 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.396345 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.396533 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.397592 (+ 1059us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:32.397602 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:32.397941 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 15:59:32.398253 (+ 312us) client_negotiation.cc:240] Negotiation successful
0806 15:59:32.398627 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":616,"spinlock_wait_cycles":2304,"thread_start_us":136,"threads_started":1}
I20260806 15:59:32.400496 996 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [sys.catalog]: This master's current role is: LEARNER
I20260806 15:59:32.401679 982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.382848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:32931 (local address 127.25.31.131:55465)
0806 15:59:32.383147 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:32.383164 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:32.383623 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:32.384187 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:32.384197 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:32.384799 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:32.385560 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.385580 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.386757 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.386760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.388486 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.388498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.389240 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.389244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.389577 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.390640 (+ 1063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:32.390653 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:32.394674 (+ 4021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:32.398330 (+ 3656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.398333 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.398335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.398601 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.398919 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.398921 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.398923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.399011 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.399470 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:32.399472 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:32.399583 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 15:59:32.400218 (+ 635us) client_negotiation.cc:240] Negotiation successful
0806 15:59:32.400362 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":505,"spinlock_wait_cycles":2432,"thread_start_us":88,"threads_started":1}
I20260806 15:59:32.402146 976 heartbeater.cc:347] Connected to a master server at 127.25.31.188:32771
I20260806 15:59:32.402468 976 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:32.402698 978 heartbeater.cc:347] Connected to a master server at 127.25.31.189:32931
I20260806 15:59:32.402743 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.391240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32886 (local address 127.25.31.189:32931)
0806 15:59:32.391516 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.391523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.391823 (+ 300us) server_negotiation.cc:407] Connection header received
0806 15:59:32.391871 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.391876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.391926 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.392021 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:32.393207 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.393999 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.395311 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.395700 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.395813 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.398721 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.398725 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.398727 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.398772 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.400594 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.401075 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.401079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.401081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.401166 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.401514 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.401520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.401524 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.401767 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.401906 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.402422 (+ 516us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.402558 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"spinlock_wait_cycles":4864,"thread_start_us":94,"threads_started":1}
I20260806 15:59:32.402781 978 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:32.403105 976 heartbeater.cc:511] Master 127.25.31.188:32771 requested a full tablet report, sending...
I20260806 15:59:32.403714 978 heartbeater.cc:511] Master 127.25.31.189:32931 requested a full tablet report, sending...
Aug 06 15:59:32 dist-test-slave-tm1g krb5kdc[32110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031972, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
I20260806 15:59:32.404593 501 ts_manager.cc:194] Registered new tserver with Master: 8b17d836eb794e359e1e0ceecb09f970 (127.25.31.131:37269)
I20260806 15:59:32.404640 315 ts_manager.cc:194] Registered new tserver with Master: 8b17d836eb794e359e1e0ceecb09f970 (127.25.31.131:37269)
I20260806 15:59:32.408205 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.381235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:37389 (local address 127.25.31.190:40393)
0806 15:59:32.381371 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.381376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.382427 (+ 1051us) server_negotiation.cc:407] Connection header received
0806 15:59:32.383903 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.383907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.383963 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.384021 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:32.385738 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.386606 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.390079 (+ 3473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.390405 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.390509 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.404248 (+ 13739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.404252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.404255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.404283 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.406414 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.406927 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.406931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.406933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.407022 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.407310 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.407315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.407316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.407563 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.407664 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.407947 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.408059 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 15:59:32.408224 981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.381587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.131:37389)
0806 15:59:32.382245 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:32.382332 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:32.383623 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:32.384187 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:32.384197 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:32.384710 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:32.385560 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.385580 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.386757 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.386760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.389952 (+ 3192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.389962 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.396441 (+ 6479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.396445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.396547 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.399757 (+ 3210us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:32.399781 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:32.404114 (+ 4333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:32.406511 (+ 2397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.406515 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.406517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.406780 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.407127 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:32.407130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:32.407132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:32.407220 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:32.407670 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:32.407674 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:32.407769 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 15:59:32.407890 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 15:59:32.408047 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":365,"spinlock_wait_cycles":18304,"thread_start_us":135,"threads_started":1}
I20260806 15:59:32.408957 993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.391871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57634 (local address 127.25.31.190:40393)
0806 15:59:32.392459 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.392470 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.392484 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:59:32.392530 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.392533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.392574 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.392661 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:32.393446 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.394210 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.395444 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.395785 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.396238 (+ 453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.401063 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.401070 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.401076 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.401105 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.406263 (+ 5158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.406889 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.406894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.406897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.406994 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.407546 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.407552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.407555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.407794 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.408571 (+ 777us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.408658 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.408783 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":65,"threads_started":1}
I20260806 15:59:32.409029 977 heartbeater.cc:347] Connected to a master server at 127.25.31.190:40393
I20260806 15:59:32.409122 977 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:32.409322 977 heartbeater.cc:511] Master 127.25.31.190:40393 requested a full tablet report, sending...
I20260806 15:59:32.409958 32156 ts_manager.cc:194] Re-registered known tserver with Master: 8b17d836eb794e359e1e0ceecb09f970 (127.25.31.131:37269)
I20260806 15:59:32.410750 32156 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:37389
I20260806 15:59:32.411441 992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.391634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39570 (local address 127.25.31.188:32771)
0806 15:59:32.393569 (+ 1935us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.393574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.393585 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:59:32.393618 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.393620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.393660 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.393741 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:32.395272 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.396092 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.405210 (+ 9118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.405615 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.405715 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.407357 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.407361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.407364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.407386 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.409234 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.409736 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.409739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.409741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.409828 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.410136 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.410139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.410141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.410436 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.410565 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.411180 (+ 615us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.411279 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1859,"thread_start_us":76,"threads_started":1}
I20260806 15:59:32.411827 32156 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:59:32.414853 32156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:32.417424 983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: NON_VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: true } } }
I20260806 15:59:32.417557 983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:32.420643 983 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006: attempting to promote NON_VOTER dbe6087f9b104582ba83f141afa0b2ef to VOTER
I20260806 15:59:32.421029 983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } }
I20260806 15:59:32.422150 515 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 1 LEARNER]: Refusing update from remote peer 13d4ef44d33d4e888756a75470614006: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 15:59:32.422245 326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 FOLLOWER]: Refusing update from remote peer 13d4ef44d33d4e888756a75470614006: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 15:59:32.422672 983 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 15:59:32.422935 983 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 15:59:32.424700 998 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, dbe6087f9b104582ba83f141afa0b2ef (127.25.31.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } } }
I20260806 15:59:32.425079 326 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, dbe6087f9b104582ba83f141afa0b2ef (127.25.31.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } } }
I20260806 15:59:32.425158 515 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, dbe6087f9b104582ba83f141afa0b2ef (127.25.31.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } } }
I20260806 15:59:32.426456 998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } } }
I20260806 15:59:32.426582 998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:32.426764 998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } } }
I20260806 15:59:32.426759 1007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } } }
I20260806 15:59:32.426872 998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:32.426872 1007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:59:32.427132 998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } } }
I20260806 15:59:32.427167 996 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13d4ef44d33d4e888756a75470614006" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } } }
I20260806 15:59:32.427274 996 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:59:32.427356 998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:32.440100 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.432302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:37269 (local address 127.0.0.1:37004)
0806 15:59:32.433073 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:32.433088 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:32.433175 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:32.435500 (+ 2325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:32.435503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:32.435522 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:32.435637 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.435643 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.437251 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.437254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.438508 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.438518 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.439672 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.439676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.439792 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.439821 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 15:59:32.439881 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 15:59:32.439952 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":2,"thread_start_us":108,"threads_started":1}
I20260806 15:59:32.440785 1008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.432538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:41197 (local address 127.0.0.1:54456)
0806 15:59:32.435182 (+ 2644us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:32.435194 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:32.435332 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:32.436010 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:32.436013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:32.436033 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:32.436130 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.436138 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.437709 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.437712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.438881 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.438890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.440152 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.440156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.440471 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.440494 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 15:59:32.440594 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 15:59:32.440663 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2584,"thread_start_us":101,"threads_started":1}
I20260806 15:59:32.440892 1010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.432952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54456 (local address 127.25.31.129:41197)
0806 15:59:32.433446 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.433463 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.435320 (+ 1857us) server_negotiation.cc:407] Connection header received
0806 15:59:32.435461 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.435466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.435676 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.435843 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:32.436266 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.437577 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.439028 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.439811 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.439919 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.440095 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.440633 (+ 538us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.440739 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":105,"threads_started":1}
I20260806 15:59:32.440986 1011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.434632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37004 (local address 127.25.31.131:37269)
0806 15:59:32.434982 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.434994 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.435018 (+ 24us) server_negotiation.cc:407] Connection header received
0806 15:59:32.435098 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.435103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.435335 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.435781 (+ 446us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:32.435944 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.437124 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.438665 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.439535 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.440543 (+ 1008us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.440688 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.440734 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.440826 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":116,"threads_started":1}
I20260806 15:59:32.442437 984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.432070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:43327 (local address 127.0.0.1:38774)
0806 15:59:32.432481 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:32.432496 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:32.432598 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:32.437011 (+ 4413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:32.437014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:32.437035 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:32.437115 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.437122 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.438641 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.438644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.440394 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.440403 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.442081 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.442084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.442173 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.442194 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 15:59:32.442236 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 15:59:32.442325 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":645}
I20260806 15:59:32.443113 1012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.434748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38774 (local address 127.25.31.130:43327)
0806 15:59:32.435592 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.435604 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.435625 (+ 21us) server_negotiation.cc:407] Connection header received
0806 15:59:32.435695 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.435699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.435866 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.437262 (+ 1396us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:32.437397 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.438513 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.441152 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.441947 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.442679 (+ 732us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.442817 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.442875 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.442963 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":116,"threads_started":1}
I20260806 15:59:32.444206 904 tablet_service.cc:1467] Processing CreateTablet for tablet a881403dbeea440e87283bdb64ccf29d (DEFAULT_TABLE table=test-table [id=bb14d7aca43842b8b7d6d1aee1b2ea30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:32.444207 620 tablet_service.cc:1467] Processing CreateTablet for tablet a881403dbeea440e87283bdb64ccf29d (DEFAULT_TABLE table=test-table [id=bb14d7aca43842b8b7d6d1aee1b2ea30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:32.445344 904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a881403dbeea440e87283bdb64ccf29d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:32.445344 620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a881403dbeea440e87283bdb64ccf29d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:32.448637 761 tablet_service.cc:1467] Processing CreateTablet for tablet a881403dbeea440e87283bdb64ccf29d (DEFAULT_TABLE table=test-table [id=bb14d7aca43842b8b7d6d1aee1b2ea30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:32.449481 761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a881403dbeea440e87283bdb64ccf29d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:32.453604 1014 tablet_bootstrap.cc:492] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970: Bootstrap starting.
I20260806 15:59:32.454052 1013 tablet_bootstrap.cc:492] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b: Bootstrap starting.
I20260806 15:59:32.456425 1014 tablet_bootstrap.cc:654] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:32.456435 1013 tablet_bootstrap.cc:654] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:32.456984 1015 tablet_bootstrap.cc:492] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d: Bootstrap starting.
I20260806 15:59:32.457317 1014 log.cc:824] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:32.457320 1013 log.cc:824] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:32.459362 1015 tablet_bootstrap.cc:654] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:32.459580 1014 tablet_bootstrap.cc:492] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970: No bootstrap required, opened a new log
I20260806 15:59:32.459781 1014 ts_tablet_manager.cc:1397] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 15:59:32.459969 1013 tablet_bootstrap.cc:492] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b: No bootstrap required, opened a new log
I20260806 15:59:32.460151 1013 ts_tablet_manager.cc:1397] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 15:59:32.460261 1015 log.cc:824] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:32.462134 1015 tablet_bootstrap.cc:492] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d: No bootstrap required, opened a new log
I20260806 15:59:32.462347 1015 ts_tablet_manager.cc:1397] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 15:59:32.463771 1014 raft_consensus.cc:348] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } }
I20260806 15:59:32.464001 1014 raft_consensus.cc:374] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:32.464056 1014 raft_consensus.cc:729] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b17d836eb794e359e1e0ceecb09f970, State: Initialized, Role: FOLLOWER
I20260806 15:59:32.464515 1014 consensus_queue.cc:260] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } }
I20260806 15:59:32.464622 1013 raft_consensus.cc:348] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } }
I20260806 15:59:32.464864 1013 raft_consensus.cc:374] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:32.464932 1013 raft_consensus.cc:729] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfad7b8c48ec447ca43f3dc53e5cf34b, State: Initialized, Role: FOLLOWER
I20260806 15:59:32.465402 1013 consensus_queue.cc:260] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } }
I20260806 15:59:32.465656 1014 ts_tablet_manager.cc:1428] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 15:59:32.465937 977 heartbeater.cc:503] Master 127.25.31.190:40393 was elected leader, sending a full tablet report...
I20260806 15:59:32.466270 1013 ts_tablet_manager.cc:1428] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 15:59:32.466650 691 heartbeater.cc:503] Master 127.25.31.190:40393 was elected leader, sending a full tablet report...
I20260806 15:59:32.470887 1015 raft_consensus.cc:348] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } }
I20260806 15:59:32.471081 1015 raft_consensus.cc:374] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:32.471155 1015 raft_consensus.cc:729] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d0f242a13004c89911dcd52f996080d, State: Initialized, Role: FOLLOWER
I20260806 15:59:32.471536 1015 consensus_queue.cc:260] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } }
I20260806 15:59:32.472150 833 heartbeater.cc:503] Master 127.25.31.190:40393 was elected leader, sending a full tablet report...
I20260806 15:59:32.472391 1015 ts_tablet_manager.cc:1428] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d: Time spent starting tablet: real 0.010s user 0.002s sys 0.003s
W20260806 15:59:32.494956 693 tablet.cc:2369] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:59:32.630695 979 tablet.cc:2369] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:59:32.673975 1020 raft_consensus.cc:482] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:32.674188 1020 raft_consensus.cc:504] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } }
I20260806 15:59:32.675320 1020 leader_election.cc:288] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d0f242a13004c89911dcd52f996080d (127.25.31.130:43327), 8b17d836eb794e359e1e0ceecb09f970 (127.25.31.131:37269)
I20260806 15:59:32.679384 1011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.676159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:43125 (local address 127.25.31.131:37269)
0806 15:59:32.676285 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.676290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.676306 (+ 16us) server_negotiation.cc:407] Connection header received
0806 15:59:32.676406 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.676410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.676462 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.676546 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:32.676839 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.677338 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.678440 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.678833 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.678925 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.679001 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.679176 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.679259 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 15:59:32.679423 1012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.675732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:53225 (local address 127.25.31.130:43327)
0806 15:59:32.675883 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.675888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.676018 (+ 130us) server_negotiation.cc:407] Connection header received
0806 15:59:32.676213 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.676219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.676272 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.676325 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:32.676793 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.677342 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.678452 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.678865 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.678996 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.679048 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.679244 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.679317 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 15:59:32.679406 1023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.676014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:37269 (local address 127.25.31.129:43125)
0806 15:59:32.676233 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:32.676245 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:32.676319 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:32.676552 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:32.676554 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:32.676567 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:32.676707 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.676711 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.677441 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.677446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.678277 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.678285 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.678945 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.678952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.679048 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.679077 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 15:59:32.679144 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 15:59:32.679223 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":21,"thread_start_us":90,"threads_started":1}
I20260806 15:59:32.679410 1022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.675635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:43327 (local address 127.25.31.129:53225)
0806 15:59:32.675901 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:32.675918 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:32.676067 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:32.676485 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:32.676490 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:32.676515 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:32.676673 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.676681 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.677467 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.677470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.678294 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:32.678318 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.678997 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.679001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:32.679105 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.679137 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 15:59:32.679217 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 15:59:32.679282 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":71,"thread_start_us":82,"threads_started":1}
I20260806 15:59:32.680207 784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a881403dbeea440e87283bdb64ccf29d" candidate_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d0f242a13004c89911dcd52f996080d" is_pre_election: true
I20260806 15:59:32.680188 926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a881403dbeea440e87283bdb64ccf29d" candidate_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b17d836eb794e359e1e0ceecb09f970" is_pre_election: true
I20260806 15:59:32.680517 926 raft_consensus.cc:2454] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfad7b8c48ec447ca43f3dc53e5cf34b in term 0.
I20260806 15:59:32.680517 784 raft_consensus.cc:2454] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfad7b8c48ec447ca43f3dc53e5cf34b in term 0.
I20260806 15:59:32.680968 572 leader_election.cc:302] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b17d836eb794e359e1e0ceecb09f970, dfad7b8c48ec447ca43f3dc53e5cf34b; no voters:
I20260806 15:59:32.681200 1020 raft_consensus.cc:2781] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:32.681294 1020 raft_consensus.cc:482] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:32.681353 1020 raft_consensus.cc:3037] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:32.682476 1020 raft_consensus.cc:504] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } }
I20260806 15:59:32.682837 1020 leader_election.cc:288] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [CANDIDATE]: Term 1 election: Requested vote from peers 1d0f242a13004c89911dcd52f996080d (127.25.31.130:43327), 8b17d836eb794e359e1e0ceecb09f970 (127.25.31.131:37269)
I20260806 15:59:32.683171 784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a881403dbeea440e87283bdb64ccf29d" candidate_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d0f242a13004c89911dcd52f996080d"
I20260806 15:59:32.683216 926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a881403dbeea440e87283bdb64ccf29d" candidate_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b17d836eb794e359e1e0ceecb09f970"
I20260806 15:59:32.683303 784 raft_consensus.cc:3037] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:32.683336 926 raft_consensus.cc:3037] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:32.684399 926 raft_consensus.cc:2454] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfad7b8c48ec447ca43f3dc53e5cf34b in term 1.
I20260806 15:59:32.684476 784 raft_consensus.cc:2454] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfad7b8c48ec447ca43f3dc53e5cf34b in term 1.
W20260806 15:59:32.684681 835 tablet.cc:2369] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:59:32.684727 572 leader_election.cc:302] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b17d836eb794e359e1e0ceecb09f970, dfad7b8c48ec447ca43f3dc53e5cf34b; no voters:
I20260806 15:59:32.684903 1020 raft_consensus.cc:2781] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:32.685086 1020 raft_consensus.cc:686] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [term 1 LEADER]: Becoming Leader. State: Replica: dfad7b8c48ec447ca43f3dc53e5cf34b, State: Running, Role: LEADER
I20260806 15:59:32.685428 1020 consensus_queue.cc:237] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } }
I20260806 15:59:32.688881 32155 catalog_manager.cc:5495] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b reported cstate change: term changed from 0 to 1, leader changed from <none> to dfad7b8c48ec447ca43f3dc53e5cf34b (127.25.31.129). New cstate: current_term: 1 leader_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } health_report { overall_health: HEALTHY } } }
I20260806 15:59:32.727972 1010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.724469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54464 (local address 127.25.31.129:41197)
0806 15:59:32.724610 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.724613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.724628 (+ 15us) server_negotiation.cc:407] Connection header received
0806 15:59:32.724735 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.724737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.724795 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.724885 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:32.725155 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.725672 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.726560 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.726769 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.726848 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.727145 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:32.727548 (+ 403us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:32.727642 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.727745 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.727800 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 15:59:32.733893 926 raft_consensus.cc:1261] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970 [term 1 FOLLOWER]: Refusing update from remote peer dfad7b8c48ec447ca43f3dc53e5cf34b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:32.733932 784 raft_consensus.cc:1261] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d [term 1 FOLLOWER]: Refusing update from remote peer dfad7b8c48ec447ca43f3dc53e5cf34b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:32.734589 1020 consensus_queue.cc:1035] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:32.734798 1024 consensus_queue.cc:1035] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:32.746539 1028 mvcc.cc:205] Tried to move back new op lower bound from 7315586960310583296 to 7315586960121901056. Current Snapshot: MvccSnapshot[applied={T|T < 7315586960310583296}]
I20260806 15:59:32.757279 32155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:32.757491 32155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:32.761878 32155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13d4ef44d33d4e888756a75470614006: Sending DeleteTablet for 3 replicas of tablet a881403dbeea440e87283bdb64ccf29d
I20260806 15:59:32.762758 620 tablet_service.cc:1514] Processing DeleteTablet for tablet a881403dbeea440e87283bdb64ccf29d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:32 UTC) from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:54456
I20260806 15:59:32.763305 1036 tablet_replica.cc:330] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b: stopping tablet replica
I20260806 15:59:32.763500 904 tablet_service.cc:1514] Processing DeleteTablet for tablet a881403dbeea440e87283bdb64ccf29d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:32 UTC) from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:37004
I20260806 15:59:32.763659 1036 raft_consensus.cc:2229] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:59:32.764101 1036 raft_consensus.cc:2258] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:32.764696 761 tablet_service.cc:1514] Processing DeleteTablet for tablet a881403dbeea440e87283bdb64ccf29d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:32 UTC) from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:38774
I20260806 15:59:32.765691 1037 tablet_replica.cc:330] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970: stopping tablet replica
I20260806 15:59:32.765980 1036 ts_tablet_manager.cc:1905] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:32.766119 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 32123
I20260806 15:59:32.766340 1039 tablet_replica.cc:330] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d: stopping tablet replica
I20260806 15:59:32.767165 1039 raft_consensus.cc:2229] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:32.767634 1039 raft_consensus.cc:2258] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:32.769573 1039 ts_tablet_manager.cc:1905] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:32.770121 1036 ts_tablet_manager.cc:1918] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:32.770231 1036 log.cc:1196] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-0/wal/wals/a881403dbeea440e87283bdb64ccf29d
I20260806 15:59:32.770605 1036 ts_tablet_manager.cc:1939] T a881403dbeea440e87283bdb64ccf29d P dfad7b8c48ec447ca43f3dc53e5cf34b: Deleting consensus metadata
I20260806 15:59:32.773247 1039 ts_tablet_manager.cc:1918] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:32.773388 1039 log.cc:1196] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/a881403dbeea440e87283bdb64ccf29d
I20260806 15:59:32.773752 1039 ts_tablet_manager.cc:1939] T a881403dbeea440e87283bdb64ccf29d P 1d0f242a13004c89911dcd52f996080d: Deleting consensus metadata
I20260806 15:59:32.775249 1037 raft_consensus.cc:2229] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:32.775593 1037 raft_consensus.cc:2258] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:32.777123 1037 ts_tablet_manager.cc:1905] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 15:59:32.777472 572 connection.cc:535] server connection from 127.0.0.1:54456 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 15:59:32.778081 716 connection.cc:535] server connection from 127.0.0.1:38774 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 15:59:32.779839 1037 ts_tablet_manager.cc:1918] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:32.779949 1037 log.cc:1196] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/a881403dbeea440e87283bdb64ccf29d
I20260806 15:59:32.780263 1037 ts_tablet_manager.cc:1939] T a881403dbeea440e87283bdb64ccf29d P 8b17d836eb794e359e1e0ceecb09f970: Deleting consensus metadata
W20260806 15:59:32.782106 856 connection.cc:441] server connection from 127.0.0.1:37004 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37004 (request call id 1) could send its response
I20260806 15:59:32.786546 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.765059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32894 (local address 127.25.31.189:32931)
0806 15:59:32.765176 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.765181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.765513 (+ 332us) server_negotiation.cc:407] Connection header received
0806 15:59:32.775133 (+ 9620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.775139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.775206 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.775290 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:32.776606 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.777496 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.778845 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.779222 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.779314 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.782160 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.782165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.782168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.782197 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.783999 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.784514 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.784518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.784520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.784610 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.784927 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.784931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.784933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.785125 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.785264 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.786185 (+ 921us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.786342 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 15:59:32.793535 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.783241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32906 (local address 127.25.31.189:32931)
0806 15:59:32.783386 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.783391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.783847 (+ 456us) server_negotiation.cc:407] Connection header received
0806 15:59:32.784197 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.784200 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.784247 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.784296 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:32.785229 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.785759 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.787268 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.787607 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.787677 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.789841 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.789844 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.789847 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.789872 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.791709 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.792199 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.792203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.792205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.792286 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.792585 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.792588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.792590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.792809 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.792910 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.793260 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.793373 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 15:59:32.795632 992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.765655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39582 (local address 127.25.31.188:32771)
0806 15:59:32.765867 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.765872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.786668 (+ 20796us) server_negotiation.cc:407] Connection header received
0806 15:59:32.786857 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.786863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.786917 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.786968 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:32.787810 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.788529 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.789427 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.789737 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.789847 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.791578 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.791582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.791585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.791611 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.793343 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.794252 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.794256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.794258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.794372 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.794683 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.794686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.794688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.794885 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.794992 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.795323 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.795457 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 15:59:32.799332 985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:32.783923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39596 (local address 127.25.31.188:32771)
0806 15:59:32.784061 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:32.784065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:32.790628 (+ 6563us) server_negotiation.cc:407] Connection header received
0806 15:59:32.790815 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:32.790819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:32.790881 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:32.790933 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:32.791890 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.792659 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.793775 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:32.794094 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:32.794155 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:32.795997 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:32.796000 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:32.796011 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:32.796032 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:32.797541 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.797988 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.797991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.797992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.798070 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:32.798485 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:32.798491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:32.798496 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:32.798684 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:32.798802 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:32.799052 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 15:59:32.799150 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 15:59:33.748286 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:33.747644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.130:46985)
0806 15:59:33.748028 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:33.748150 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":80,"threads_started":1}
W20260806 15:59:33.748935 833 heartbeater.cc:644] Failed to heartbeat to 127.25.31.190:40393 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
I20260806 15:59:33.750290 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:33.749580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.129:34445)
0806 15:59:33.750101 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:33.750193 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":65,"threads_started":1}
I20260806 15:59:33.750470 1049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:33.750023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.131:41035)
0806 15:59:33.750252 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:33.750361 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":69,"threads_started":1}
W20260806 15:59:33.750797 691 heartbeater.cc:644] Failed to heartbeat to 127.25.31.190:40393 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
W20260806 15:59:33.750921 977 heartbeater.cc:644] Failed to heartbeat to 127.25.31.190:40393 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
I20260806 15:59:34.262296 1050 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 13d4ef44d33d4e888756a75470614006)
I20260806 15:59:34.262341 1051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 13d4ef44d33d4e888756a75470614006)
I20260806 15:59:34.262475 1051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } }
I20260806 15:59:34.262568 1050 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } }
I20260806 15:59:34.263056 1051 leader_election.cc:288] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13d4ef44d33d4e888756a75470614006 (127.25.31.190:40393), dbe6087f9b104582ba83f141afa0b2ef (127.25.31.188:32771)
I20260806 15:59:34.263741 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:34.263173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:58092)
0806 15:59:34.263514 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:34.263619 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":62,"threads_started":1}
I20260806 15:59:34.263705 1050 leader_election.cc:288] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13d4ef44d33d4e888756a75470614006 (127.25.31.190:40393), 7bb3bdf4e8a24987b424f27bc1daa914 (127.25.31.189:32931)
W20260806 15:59:34.264074 300 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
I20260806 15:59:34.264134 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:34.263784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:58100)
0806 15:59:34.263976 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:34.264046 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
W20260806 15:59:34.264539 490 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
I20260806 15:59:34.265275 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:34.265034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:58102)
0806 15:59:34.265167 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:34.265213 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260806 15:59:34.265650 300 leader_election.cc:334] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 13d4ef44d33d4e888756a75470614006 (127.25.31.190:40393): Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
I20260806 15:59:34.265801 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:34.265525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:58114)
0806 15:59:34.265630 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:34.265683 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 15:59:34.266008 490 leader_election.cc:334] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 13d4ef44d33d4e888756a75470614006 (127.25.31.190:40393): Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
I20260806 15:59:34.268194 1053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:34.263273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:32771 (local address 127.0.0.1:39600)
0806 15:59:34.263619 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:34.263635 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:34.264294 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:34.264660 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:34.264664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:34.264684 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:34.264793 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:34.264801 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:34.265935 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:34.265939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:34.266881 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:34.266890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:34.267755 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:34.267761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:34.267870 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:34.267896 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 15:59:34.267983 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 15:59:34.268055 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":56,"threads_started":1}
I20260806 15:59:34.268217 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:34.264005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32908 (local address 127.25.31.189:32931)
0806 15:59:34.264261 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:34.264266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:34.264404 (+ 138us) server_negotiation.cc:407] Connection header received
0806 15:59:34.264531 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:34.264534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:34.264582 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:34.264660 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:34.265058 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:34.265823 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:34.266997 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:34.267597 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:34.267713 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:34.267779 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:34.268020 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 15:59:34.268092 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":77,"threads_started":1}
I20260806 15:59:34.268224 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:34.263878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:32931 (local address 127.0.0.1:32908)
0806 15:59:34.264261 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:34.264278 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:34.264394 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:34.264787 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:34.264793 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:34.264826 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:34.264931 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:34.264940 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:34.265971 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:34.265975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:34.266885 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:34.266894 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:34.267741 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:34.267745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:34.267881 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:34.267903 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 15:59:34.267983 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 15:59:34.268055 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":32,"thread_start_us":70,"threads_started":1}
I20260806 15:59:34.268224 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:34.263435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39600 (local address 127.25.31.188:32771)
0806 15:59:34.264398 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:34.264403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:34.264416 (+ 13us) server_negotiation.cc:407] Connection header received
0806 15:59:34.264463 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:34.264465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:34.264530 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:34.264981 (+ 451us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:34.265080 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:34.265805 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:34.267037 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:34.267633 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:34.267726 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:34.267802 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:34.268009 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 15:59:34.268087 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":893,"thread_start_us":72,"threads_started":1}
I20260806 15:59:34.268743 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbe6087f9b104582ba83f141afa0b2ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" is_pre_election: true
I20260806 15:59:34.268744 515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "dbe6087f9b104582ba83f141afa0b2ef" is_pre_election: true
I20260806 15:59:34.268944 326 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe6087f9b104582ba83f141afa0b2ef in term 1.
I20260806 15:59:34.269002 515 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bb3bdf4e8a24987b424f27bc1daa914 in term 1.
I20260806 15:59:34.269299 489 leader_election.cc:302] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bb3bdf4e8a24987b424f27bc1daa914, dbe6087f9b104582ba83f141afa0b2ef; no voters: 13d4ef44d33d4e888756a75470614006
I20260806 15:59:34.269400 32767 leader_election.cc:302] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bb3bdf4e8a24987b424f27bc1daa914, dbe6087f9b104582ba83f141afa0b2ef; no voters: 13d4ef44d33d4e888756a75470614006
I20260806 15:59:34.269521 1050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 15:59:34.269552 1051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 15:59:34.269583 1050 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 1 FOLLOWER]: Starting leader election (detected failure of leader 13d4ef44d33d4e888756a75470614006)
I20260806 15:59:34.269603 1051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 13d4ef44d33d4e888756a75470614006)
I20260806 15:59:34.269629 1050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 1 FOLLOWER]: Advancing to term 2
I20260806 15:59:34.269652 1051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 1 FOLLOWER]: Advancing to term 2
I20260806 15:59:34.270915 1050 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } }
I20260806 15:59:34.270915 1051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } }
I20260806 15:59:34.271289 1050 leader_election.cc:288] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [CANDIDATE]: Term 2 election: Requested vote from peers 13d4ef44d33d4e888756a75470614006 (127.25.31.190:40393), 7bb3bdf4e8a24987b424f27bc1daa914 (127.25.31.189:32931)
I20260806 15:59:34.271294 1051 leader_election.cc:288] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [CANDIDATE]: Term 2 election: Requested vote from peers 13d4ef44d33d4e888756a75470614006 (127.25.31.190:40393), dbe6087f9b104582ba83f141afa0b2ef (127.25.31.188:32771)
I20260806 15:59:34.271616 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:34.271400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:58128)
0806 15:59:34.271500 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:34.271558 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 15:59:34.271677 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbe6087f9b104582ba83f141afa0b2ef" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7bb3bdf4e8a24987b424f27bc1daa914"
I20260806 15:59:34.271808 1053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:34.271411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:58130)
0806 15:59:34.271688 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:34.271742 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
I20260806 15:59:34.271821 326 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate dbe6087f9b104582ba83f141afa0b2ef in current term 2: Already voted for candidate 7bb3bdf4e8a24987b424f27bc1daa914 in this term.
I20260806 15:59:34.271896 515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "dbe6087f9b104582ba83f141afa0b2ef"
I20260806 15:59:34.272046 515 raft_consensus.cc:2379] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7bb3bdf4e8a24987b424f27bc1daa914 in current term 2: Already voted for candidate dbe6087f9b104582ba83f141afa0b2ef in this term.
I20260806 15:59:34.272411 1053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:34.272218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:58144)
0806 15:59:34.272317 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:34.272365 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 15:59:34.272511 300 leader_election.cc:334] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 13d4ef44d33d4e888756a75470614006 (127.25.31.190:40393): Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
I20260806 15:59:34.272614 32767 leader_election.cc:302] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7bb3bdf4e8a24987b424f27bc1daa914; no voters: 13d4ef44d33d4e888756a75470614006, dbe6087f9b104582ba83f141afa0b2ef
I20260806 15:59:34.272668 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:34.272366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:58152)
0806 15:59:34.272529 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:34.272604 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 15:59:34.272776 490 leader_election.cc:334] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 13d4ef44d33d4e888756a75470614006 (127.25.31.190:40393): Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
I20260806 15:59:34.272894 490 leader_election.cc:302] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dbe6087f9b104582ba83f141afa0b2ef; no voters: 13d4ef44d33d4e888756a75470614006, 7bb3bdf4e8a24987b424f27bc1daa914
I20260806 15:59:34.272921 1051 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 15:59:34.273058 1050 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 15:59:34.750075 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:34.749621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.130:53985)
0806 15:59:34.749873 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:34.749936 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":77,"threads_started":1}
I20260806 15:59:34.751870 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:34.751424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.129:35163)
0806 15:59:34.751704 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:34.751764 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":116,"threads_started":1}
I20260806 15:59:34.751885 1063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:34.751416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.131:58607)
0806 15:59:34.751758 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:34.751806 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":120,"threads_started":1}
I20260806 15:59:35.751423 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:35.750906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.130:46561)
0806 15:59:35.751175 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:35.751283 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":74,"threads_started":1}
I20260806 15:59:35.752960 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:35.752518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.131:51869)
0806 15:59:35.752798 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:35.752859 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":65,"threads_started":1}
I20260806 15:59:35.753443 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:35.752774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.129:46953)
0806 15:59:35.753282 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:35.753339 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":103,"threads_started":1}
I20260806 15:59:35.994228 1067 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:35.994395 1067 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } }
I20260806 15:59:35.994885 1067 leader_election.cc:288] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 13d4ef44d33d4e888756a75470614006 (127.25.31.190:40393), 7bb3bdf4e8a24987b424f27bc1daa914 (127.25.31.189:32931)
I20260806 15:59:35.995361 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbe6087f9b104582ba83f141afa0b2ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" is_pre_election: true
I20260806 15:59:35.995437 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:35.995095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:58356)
0806 15:59:35.995303 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:35.995359 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
I20260806 15:59:35.995517 326 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe6087f9b104582ba83f141afa0b2ef in term 2.
I20260806 15:59:35.995839 489 leader_election.cc:302] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bb3bdf4e8a24987b424f27bc1daa914, dbe6087f9b104582ba83f141afa0b2ef; no voters:
I20260806 15:59:35.996014 1067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 15:59:35.996084 1067 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:35.996126 1067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 2 FOLLOWER]: Advancing to term 3
I20260806 15:59:35.996315 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:35.996102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:58368)
0806 15:59:35.996209 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:35.996262 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 15:59:35.996484 490 leader_election.cc:334] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 13d4ef44d33d4e888756a75470614006 (127.25.31.190:40393): Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
I20260806 15:59:35.997325 1067 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } }
I20260806 15:59:35.997644 1067 leader_election.cc:288] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [CANDIDATE]: Term 3 election: Requested vote from peers 13d4ef44d33d4e888756a75470614006 (127.25.31.190:40393), 7bb3bdf4e8a24987b424f27bc1daa914 (127.25.31.189:32931)
I20260806 15:59:35.997929 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:35.997722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:58376)
0806 15:59:35.997815 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:35.997864 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 15:59:35.998039 326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dbe6087f9b104582ba83f141afa0b2ef" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7bb3bdf4e8a24987b424f27bc1daa914"
I20260806 15:59:35.998174 326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 2 FOLLOWER]: Advancing to term 3
I20260806 15:59:35.998620 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:35.998419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:58380)
0806 15:59:35.998521 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:35.998569 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 15:59:35.998754 490 leader_election.cc:334] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 13d4ef44d33d4e888756a75470614006 (127.25.31.190:40393): Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
I20260806 15:59:35.999310 326 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe6087f9b104582ba83f141afa0b2ef in term 3.
I20260806 15:59:35.999586 489 leader_election.cc:302] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7bb3bdf4e8a24987b424f27bc1daa914, dbe6087f9b104582ba83f141afa0b2ef; no voters: 13d4ef44d33d4e888756a75470614006
I20260806 15:59:35.999754 1067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 3 FOLLOWER]: Leader election won for term 3
I20260806 15:59:35.999931 1067 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [term 3 LEADER]: Becoming Leader. State: Replica: dbe6087f9b104582ba83f141afa0b2ef, State: Running, Role: LEADER
I20260806 15:59:36.000200 1067 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } }
I20260806 15:59:36.001569 1070 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbe6087f9b104582ba83f141afa0b2ef. Latest consensus state: current_term: 3 leader_uuid: "dbe6087f9b104582ba83f141afa0b2ef" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } } }
I20260806 15:59:36.001694 1070 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:36.002036 1072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:36.004882 1072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:36.005347 1072 catalog_manager.cc:1259] Loaded cluster ID: 4eeb26800d1d43969bcf8b6df3538011
I20260806 15:59:36.005393 1072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:36.005962 1072 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:36.006390 1072 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: Loaded TSK: 0
I20260806 15:59:36.007052 1072 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:59:36.091503 501 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:59:36.094650 501 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:36.097672 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.097410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:58422)
0806 15:59:36.097536 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.097594 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 15:59:36.097739 326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [term 3 FOLLOWER]: Refusing update from remote peer dbe6087f9b104582ba83f141afa0b2ef: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 15:59:36.098255 1070 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 15:59:36.098452 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.098183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.0.0.1:58432)
0806 15:59:36.098318 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.098374 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 15:59:36.098558 490 consensus_peers.cc:487] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef -> Peer 13d4ef44d33d4e888756a75470614006 (127.25.31.190:40393): Couldn't send request to peer 13d4ef44d33d4e888756a75470614006. Status: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 15:59:36.099758 1074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbe6087f9b104582ba83f141afa0b2ef. Latest consensus state: current_term: 3 leader_uuid: "dbe6087f9b104582ba83f141afa0b2ef" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } } }
I20260806 15:59:36.099893 1074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:59:36.100929 1070 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "dbe6087f9b104582ba83f141afa0b2ef" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } } }
I20260806 15:59:36.101055 1070 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:36.103427 1074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "dbe6087f9b104582ba83f141afa0b2ef" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "13d4ef44d33d4e888756a75470614006" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40393 } } peers { permanent_uuid: "7bb3bdf4e8a24987b424f27bc1daa914" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 32931 } attrs { promote: false } } peers { permanent_uuid: "dbe6087f9b104582ba83f141afa0b2ef" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 32771 } attrs { promote: false } } }
I20260806 15:59:36.103525 1074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb3bdf4e8a24987b424f27bc1daa914 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:59:36.110893 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.107236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:37269 (local address 127.0.0.1:37020)
0806 15:59:36.107467 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.107483 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:36.107579 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:36.107916 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:36.107921 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:36.107940 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:36.108016 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:36.108020 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:36.108925 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:36.108930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:36.109741 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:36.109750 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:36.110473 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:36.110477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:36.110568 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:36.110590 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 15:59:36.110677 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 15:59:36.110752 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":20,"thread_start_us":91,"threads_started":1}
I20260806 15:59:36.110924 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.107342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37020 (local address 127.25.31.131:37269)
0806 15:59:36.107596 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:36.107602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:36.107619 (+ 17us) server_negotiation.cc:407] Connection header received
0806 15:59:36.107670 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:36.107675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:36.107723 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:36.107800 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:36.108140 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:36.108817 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:36.109865 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:36.110349 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:36.110466 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:36.110532 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:36.110709 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 15:59:36.110784 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":79,"threads_started":1}
I20260806 15:59:36.111521 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.106859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:41197 (local address 127.0.0.1:54470)
0806 15:59:36.107084 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.107096 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:36.107182 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:36.107692 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:36.107695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:36.107708 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:36.107836 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:36.107843 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:36.108983 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:36.108987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:36.110217 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:36.110227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:36.111118 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:36.111122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:36.111210 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:36.111238 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 15:59:36.111323 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 15:59:36.111397 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260806 15:59:36.111794 904 tablet_service.cc:1467] Processing CreateTablet for tablet 3490a1ced05b4ad8956c770b842cca97 (DEFAULT_TABLE table=test-table [id=59983ee7dd724ac9b3a20823fade33a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:36.112138 904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3490a1ced05b4ad8956c770b842cca97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:36.112401 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.106553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:43327 (local address 127.0.0.1:38788)
0806 15:59:36.106699 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.106717 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:36.106808 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:36.108191 (+ 1383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:36.108196 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:36.108214 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:36.108306 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:36.108312 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:36.109282 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:36.109285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:36.110416 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:36.110424 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:36.112075 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:36.112078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:36.112144 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:36.112163 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 15:59:36.112254 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 15:59:36.112320 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 15:59:36.112681 1081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.106790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38788 (local address 127.25.31.130:43327)
0806 15:59:36.107964 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:36.107968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:36.107981 (+ 13us) server_negotiation.cc:407] Connection header received
0806 15:59:36.108027 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:36.108030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:36.108073 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:36.108185 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:36.108414 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:36.109164 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:36.111228 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:36.111921 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:36.112066 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:36.112128 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:36.112321 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 15:59:36.112396 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1086,"thread_start_us":79,"threads_started":1}
I20260806 15:59:36.113188 761 tablet_service.cc:1467] Processing CreateTablet for tablet 3490a1ced05b4ad8956c770b842cca97 (DEFAULT_TABLE table=test-table [id=59983ee7dd724ac9b3a20823fade33a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:36.113480 761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3490a1ced05b4ad8956c770b842cca97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:36.114382 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.107200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54470 (local address 127.25.31.129:41197)
0806 15:59:36.107441 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:36.107446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:36.107461 (+ 15us) server_negotiation.cc:407] Connection header received
0806 15:59:36.107512 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:36.107515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:36.107562 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:36.107694 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:36.107964 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:36.108838 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:36.110400 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:36.111016 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:36.111123 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:36.111195 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:36.111343 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 15:59:36.111433 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":68,"threads_started":1}
I20260806 15:59:36.114887 620 tablet_service.cc:1467] Processing CreateTablet for tablet 3490a1ced05b4ad8956c770b842cca97 (DEFAULT_TABLE table=test-table [id=59983ee7dd724ac9b3a20823fade33a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:36.115170 620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3490a1ced05b4ad8956c770b842cca97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:36.115918 1088 tablet_bootstrap.cc:492] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d: Bootstrap starting.
I20260806 15:59:36.116828 1087 tablet_bootstrap.cc:492] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970: Bootstrap starting.
I20260806 15:59:36.116943 1088 tablet_bootstrap.cc:654] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:36.117208 1089 tablet_bootstrap.cc:492] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b: Bootstrap starting.
I20260806 15:59:36.117708 1087 tablet_bootstrap.cc:654] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:36.118114 1089 tablet_bootstrap.cc:654] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:36.118163 1088 tablet_bootstrap.cc:492] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d: No bootstrap required, opened a new log
I20260806 15:59:36.118247 1088 ts_tablet_manager.cc:1397] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:36.118645 1088 raft_consensus.cc:348] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } }
I20260806 15:59:36.118788 1088 raft_consensus.cc:374] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:36.118839 1088 raft_consensus.cc:729] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d0f242a13004c89911dcd52f996080d, State: Initialized, Role: FOLLOWER
I20260806 15:59:36.118999 1088 consensus_queue.cc:260] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } }
I20260806 15:59:36.119297 1088 ts_tablet_manager.cc:1428] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:59:36.119407 1089 tablet_bootstrap.cc:492] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b: No bootstrap required, opened a new log
I20260806 15:59:36.119485 1089 ts_tablet_manager.cc:1397] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:59:36.119968 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.119719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.130:55381)
0806 15:59:36.119832 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.119885 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 15:59:36.119887 1089 raft_consensus.cc:348] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } }
I20260806 15:59:36.120019 1089 raft_consensus.cc:374] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:36.120060 1089 raft_consensus.cc:729] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfad7b8c48ec447ca43f3dc53e5cf34b, State: Initialized, Role: FOLLOWER
I20260806 15:59:36.120204 1089 consensus_queue.cc:260] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } }
I20260806 15:59:36.120340 1087 tablet_bootstrap.cc:492] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970: No bootstrap required, opened a new log
I20260806 15:59:36.120431 1087 ts_tablet_manager.cc:1397] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 15:59:36.120617 1089 ts_tablet_manager.cc:1428] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:59:36.120850 1087 raft_consensus.cc:348] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } }
I20260806 15:59:36.120963 1087 raft_consensus.cc:374] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:36.121006 1087 raft_consensus.cc:729] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b17d836eb794e359e1e0ceecb09f970, State: Initialized, Role: FOLLOWER
I20260806 15:59:36.121129 1087 consensus_queue.cc:260] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } }
I20260806 15:59:36.121475 1087 ts_tablet_manager.cc:1428] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:59:36.123618 832 heartbeater.cc:503] Master 127.25.31.188:32771 was elected leader, sending a full tablet report...
I20260806 15:59:36.124342 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.124110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.131:50915)
0806 15:59:36.124236 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.124285 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 15:59:36.124915 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.123219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.129:53945)
0806 15:59:36.124817 (+ 1598us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.124862 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1533}
I20260806 15:59:36.125880 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.125669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.130:46831)
0806 15:59:36.125771 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.125819 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 15:59:36.210649 1094 raft_consensus.cc:482] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:36.210811 1094 raft_consensus.cc:504] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } }
I20260806 15:59:36.211225 1094 leader_election.cc:288] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d0f242a13004c89911dcd52f996080d (127.25.31.130:43327), 8b17d836eb794e359e1e0ceecb09f970 (127.25.31.131:37269)
I20260806 15:59:36.211745 784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3490a1ced05b4ad8956c770b842cca97" candidate_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d0f242a13004c89911dcd52f996080d" is_pre_election: true
I20260806 15:59:36.211777 926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3490a1ced05b4ad8956c770b842cca97" candidate_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b17d836eb794e359e1e0ceecb09f970" is_pre_election: true
I20260806 15:59:36.211908 926 raft_consensus.cc:2454] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfad7b8c48ec447ca43f3dc53e5cf34b in term 0.
I20260806 15:59:36.211910 784 raft_consensus.cc:2454] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfad7b8c48ec447ca43f3dc53e5cf34b in term 0.
I20260806 15:59:36.212239 572 leader_election.cc:302] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b17d836eb794e359e1e0ceecb09f970, dfad7b8c48ec447ca43f3dc53e5cf34b; no voters:
I20260806 15:59:36.212410 1094 raft_consensus.cc:2781] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:36.212487 1094 raft_consensus.cc:482] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:36.212535 1094 raft_consensus.cc:3037] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:36.213516 1094 raft_consensus.cc:504] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } }
I20260806 15:59:36.213824 1094 leader_election.cc:288] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [CANDIDATE]: Term 1 election: Requested vote from peers 1d0f242a13004c89911dcd52f996080d (127.25.31.130:43327), 8b17d836eb794e359e1e0ceecb09f970 (127.25.31.131:37269)
I20260806 15:59:36.214164 784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3490a1ced05b4ad8956c770b842cca97" candidate_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d0f242a13004c89911dcd52f996080d"
I20260806 15:59:36.214178 926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3490a1ced05b4ad8956c770b842cca97" candidate_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b17d836eb794e359e1e0ceecb09f970"
I20260806 15:59:36.214284 784 raft_consensus.cc:3037] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:36.214284 926 raft_consensus.cc:3037] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:36.215409 784 raft_consensus.cc:2454] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfad7b8c48ec447ca43f3dc53e5cf34b in term 1.
I20260806 15:59:36.215409 926 raft_consensus.cc:2454] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfad7b8c48ec447ca43f3dc53e5cf34b in term 1.
I20260806 15:59:36.215700 572 leader_election.cc:302] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b17d836eb794e359e1e0ceecb09f970, dfad7b8c48ec447ca43f3dc53e5cf34b; no voters:
I20260806 15:59:36.215855 1094 raft_consensus.cc:2781] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:36.216079 1094 raft_consensus.cc:686] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [term 1 LEADER]: Becoming Leader. State: Replica: dfad7b8c48ec447ca43f3dc53e5cf34b, State: Running, Role: LEADER
I20260806 15:59:36.216226 1094 consensus_queue.cc:237] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } }
I20260806 15:59:36.216315 690 heartbeater.cc:503] Master 127.25.31.188:32771 was elected leader, sending a full tablet report...
I20260806 15:59:36.216833 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.216560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.129:42975)
0806 15:59:36.216703 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.216768 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 15:59:36.218050 505 catalog_manager.cc:5495] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b reported cstate change: term changed from 0 to 1, leader changed from <none> to dfad7b8c48ec447ca43f3dc53e5cf34b (127.25.31.129). New cstate: current_term: 1 leader_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfad7b8c48ec447ca43f3dc53e5cf34b" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 } health_report { overall_health: UNKNOWN } } }
I20260806 15:59:36.238288 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.235578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54480 (local address 127.25.31.129:41197)
0806 15:59:36.235693 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:36.235696 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:36.235709 (+ 13us) server_negotiation.cc:407] Connection header received
0806 15:59:36.235801 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:36.235804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:36.235845 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:36.235908 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:36.236168 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:36.236660 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:36.237450 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:36.237652 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:36.237725 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:36.237943 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:36.238013 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:36.238064 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:36.238146 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 15:59:36.238190 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 15:59:36.240317 784 raft_consensus.cc:1261] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d [term 1 FOLLOWER]: Refusing update from remote peer dfad7b8c48ec447ca43f3dc53e5cf34b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:36.240317 926 raft_consensus.cc:1261] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970 [term 1 FOLLOWER]: Refusing update from remote peer dfad7b8c48ec447ca43f3dc53e5cf34b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:36.240757 1094 consensus_queue.cc:1035] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d0f242a13004c89911dcd52f996080d" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:36.240917 1096 consensus_queue.cc:1035] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b17d836eb794e359e1e0ceecb09f970" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:36.241930 976 heartbeater.cc:503] Master 127.25.31.188:32771 was elected leader, sending a full tablet report...
I20260806 15:59:36.243366 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.243118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.130:59959)
0806 15:59:36.243231 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.243287 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 15:59:36.243433 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.243144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.131:35289)
0806 15:59:36.243274 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.243362 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 15:59:36.246833 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.246611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.129:34619)
0806 15:59:36.246716 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.246770 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 15:59:36.246984 1100 mvcc.cc:205] Tried to move back new op lower bound from 7315586974676299776 to 7315586974583152640. Current Snapshot: MvccSnapshot[applied={T|T < 7315586974676299776}]
I20260806 15:59:36.248041 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.247755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.131:33347)
0806 15:59:36.247899 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.247956 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 15:59:36.248759 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.246515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.130:57039)
0806 15:59:36.248640 (+ 2125us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.248693 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2050}
I20260806 15:59:36.250842 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.250530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40393 (local address 127.25.31.129:41497)
0806 15:59:36.250708 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.250777 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:40393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 15:59:36.251118 505 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:36.251272 505 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:36.254743 505 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dbe6087f9b104582ba83f141afa0b2ef: Sending DeleteTablet for 3 replicas of tablet 3490a1ced05b4ad8956c770b842cca97
I20260806 15:59:36.255587 761 tablet_service.cc:1514] Processing DeleteTablet for tablet 3490a1ced05b4ad8956c770b842cca97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:36 UTC) from {username='oryx', principal='oryx/127.25.31.188@KRBTEST.COM'} at 127.0.0.1:38788
I20260806 15:59:36.255661 904 tablet_service.cc:1514] Processing DeleteTablet for tablet 3490a1ced05b4ad8956c770b842cca97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:36 UTC) from {username='oryx', principal='oryx/127.25.31.188@KRBTEST.COM'} at 127.0.0.1:37020
I20260806 15:59:36.255659 620 tablet_service.cc:1514] Processing DeleteTablet for tablet 3490a1ced05b4ad8956c770b842cca97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:36 UTC) from {username='oryx', principal='oryx/127.25.31.188@KRBTEST.COM'} at 127.0.0.1:54470
I20260806 15:59:36.255959 1107 tablet_replica.cc:330] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d: stopping tablet replica
I20260806 15:59:36.256105 1109 tablet_replica.cc:330] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b: stopping tablet replica
I20260806 15:59:36.256120 1107 raft_consensus.cc:2229] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:36.256245 1107 raft_consensus.cc:2258] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:36.256309 1109 raft_consensus.cc:2229] T 3490a1ced05b4ad8956c770b842cca97 P dfad7b8c48ec447ca43f3dc53e5cf34b [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:59:36.256515 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 465
I20260806 15:59:36.260665 1107 ts_tablet_manager.cc:1905] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:36.262470 1108 tablet_replica.cc:330] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970: stopping tablet replica
I20260806 15:59:36.262615 1108 raft_consensus.cc:2229] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:36.262755 1108 raft_consensus.cc:2258] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:36.263386 1108 ts_tablet_manager.cc:1905] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:36.264011 1107 ts_tablet_manager.cc:1918] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:36.264097 1107 log.cc:1196] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/3490a1ced05b4ad8956c770b842cca97
I20260806 15:59:36.264422 1107 ts_tablet_manager.cc:1939] T 3490a1ced05b4ad8956c770b842cca97 P 1d0f242a13004c89911dcd52f996080d: Deleting consensus metadata
I20260806 15:59:36.265252 489 catalog_manager.cc:4841] TS 1d0f242a13004c89911dcd52f996080d (127.25.31.130:43327): tablet 3490a1ced05b4ad8956c770b842cca97 (table test-table [id=59983ee7dd724ac9b3a20823fade33a6]) successfully deleted
I20260806 15:59:36.265457 1108 ts_tablet_manager.cc:1918] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:36.265558 1108 log.cc:1196] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/3490a1ced05b4ad8956c770b842cca97
I20260806 15:59:36.265795 1108 ts_tablet_manager.cc:1939] T 3490a1ced05b4ad8956c770b842cca97 P 8b17d836eb794e359e1e0ceecb09f970: Deleting consensus metadata
I20260806 15:59:36.266389 489 catalog_manager.cc:4841] TS 8b17d836eb794e359e1e0ceecb09f970 (127.25.31.131:37269): tablet 3490a1ced05b4ad8956c770b842cca97 (table test-table [id=59983ee7dd724ac9b3a20823fade33a6]) successfully deleted
I20260806 15:59:36.268162 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 698
I20260806 15:59:36.268644 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:36.268326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:41197 (local address 127.0.0.1:54484)
0806 15:59:36.268503 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:36.268558 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:41197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260806 15:59:36.269343 491 catalog_manager.cc:4584] TS dfad7b8c48ec447ca43f3dc53e5cf34b (127.25.31.129:41197): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3490a1ced05b4ad8956c770b842cca97: Network error: Client connection negotiation failed: client connection to 127.25.31.129:41197: connect: Connection refused (error 111)
I20260806 15:59:36.274910 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 839
I20260806 15:59:36.280951 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 32640
I20260806 15:59:36.287400 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 32755
2026-08-06T15:59:36Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7214 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:59:36 dist-test-slave-tm1g krb5kdc[1112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:59:36 dist-test-slave-tm1g krb5kdc[1112](info): set up 2 sockets
Aug 06 15:59:36 dist-test-slave-tm1g krb5kdc[1112](info): commencing operation
krb5kdc: starting...
W20260806 15:59:37.355161 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 15:59:37 dist-test-slave-tm1g krb5kdc[1112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:59:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.190.keytab.
Entry for principal oryx/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.190.keytab.
I20260806 15:59:37.472688 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:36251
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.190.keytab
--principal=oryx/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:33221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:36251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:37.580113 1125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:37.580351 1125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:37.580413 1125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:37.584388 1125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:37.584438 1125 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 15:59:37.584455 1125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:37.584473 1125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:37.584522 1125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:37.584550 1125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:37.589648 1125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:33221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:36251
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.190.keytab
--principal=oryx/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:36251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:37.590631 1125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:37.591606 1125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:37.599949 1131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:37.599925 1132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:37.599956 1134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:37.600183 1125 server_base.cc:1034] running on GCE node
I20260806 15:59:37.600792 1125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:37.601799 1125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:37.602982 1125 hybrid_clock.cc:648] HybridClock initialized: now 1786031977602959 us; error 38 us; skew 500 ppm
Aug 06 15:59:37 dist-test-slave-tm1g krb5kdc[1112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031977, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:37.605556 1125 init.cc:377] Logged in from keytab as oryx/127.25.31.190@KRBTEST.COM (short username oryx)
I20260806 15:59:37.606454 1125 webserver.cc:460] Webserver started at http://127.25.31.190:41539/ using document root <none> and password file <none>
I20260806 15:59:37.606976 1125 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:37.607059 1125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:37.607276 1125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:37.609212 1125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "50898636f72347dfa45c3045c902dba7"
format_stamp: "Formatted at 2026-08-06 15:59:37 on dist-test-slave-tm1g"
server_key: "7057ef0c435e7ea5294653616b9fce15"
server_key_iv: "09b34ee7e01dcde23101bb33746604af"
server_key_version: "encryptionkey@0"
I20260806 15:59:37.609762 1125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "50898636f72347dfa45c3045c902dba7"
format_stamp: "Formatted at 2026-08-06 15:59:37 on dist-test-slave-tm1g"
server_key: "7057ef0c435e7ea5294653616b9fce15"
server_key_iv: "09b34ee7e01dcde23101bb33746604af"
server_key_version: "encryptionkey@0"
I20260806 15:59:37.613238 1125 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 15:59:37.615485 1142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:37.616597 1125 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 15:59:37.616726 1125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "50898636f72347dfa45c3045c902dba7"
format_stamp: "Formatted at 2026-08-06 15:59:37 on dist-test-slave-tm1g"
server_key: "7057ef0c435e7ea5294653616b9fce15"
server_key_iv: "09b34ee7e01dcde23101bb33746604af"
server_key_version: "encryptionkey@0"
I20260806 15:59:37.616827 1125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:37.627854 1125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:37.630564 1125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:37.630740 1125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:37.638159 1125 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:36251
I20260806 15:59:37.638175 1206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:36251 every 8 connection(s)
I20260806 15:59:37.639077 1125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:37.641783 1207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:37.647264 1207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7: Bootstrap starting.
I20260806 15:59:37.647970 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 1125
I20260806 15:59:37.648089 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:37.648401 25726 external_mini_cluster.cc:1428] Setting key 5a7dc5266974548f036c794b41b5e43f
I20260806 15:59:37.650200 1207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:37.650967 1207 log.cc:824] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:37.652781 1207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7: No bootstrap required, opened a new log
I20260806 15:59:37.656190 1207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50898636f72347dfa45c3045c902dba7" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 36251 } }
I20260806 15:59:37.656458 1207 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:37.656525 1207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50898636f72347dfa45c3045c902dba7, State: Initialized, Role: FOLLOWER
Aug 06 15:59:37 dist-test-slave-tm1g krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
I20260806 15:59:37.657014 1207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50898636f72347dfa45c3045c902dba7" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 36251 } }
I20260806 15:59:37.657186 1207 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:37.657263 1207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:37.657378 1207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:37.658336 1207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50898636f72347dfa45c3045c902dba7" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 36251 } }
I20260806 15:59:37.658668 1207 leader_election.cc:302] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50898636f72347dfa45c3045c902dba7; no voters:
I20260806 15:59:37.659000 1207 leader_election.cc:288] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:37.659108 1212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:37.659309 1212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [term 1 LEADER]: Becoming Leader. State: Replica: 50898636f72347dfa45c3045c902dba7, State: Running, Role: LEADER
I20260806 15:59:37.659677 1212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50898636f72347dfa45c3045c902dba7" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 36251 } }
I20260806 15:59:37.660324 1207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:37.661770 1214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50898636f72347dfa45c3045c902dba7. Latest consensus state: current_term: 1 leader_uuid: "50898636f72347dfa45c3045c902dba7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50898636f72347dfa45c3045c902dba7" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 36251 } } }
I20260806 15:59:37.661985 1214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:37.662228 1213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50898636f72347dfa45c3045c902dba7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50898636f72347dfa45c3045c902dba7" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 36251 } } }
I20260806 15:59:37.662362 1213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:37.662545 1221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:37.663970 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:37.649660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39078 (local address 127.25.31.190:36251)
0806 15:59:37.650074 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:37.650088 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:37.650121 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:59:37.650827 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:37.650846 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:37.651207 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:37.651528 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:37.652360 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:37.653403 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:37.654256 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:37.654771 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:37.654938 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:37.657639 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:37.657644 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:37.657655 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:37.657680 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:37.659839 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:37.660331 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:37.660337 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:37.660342 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:37.660476 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:37.660804 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:37.660808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:37.660811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:37.661286 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:37.661540 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:37.662620 (+ 1080us) server_negotiation.cc:299] Negotiation successful
0806 15:59:37.662994 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":136,"threads_started":1}
I20260806 15:59:37.665619 1221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:37.670936 1221 catalog_manager.cc:1347] Generated new cluster ID: 0675e3cec7614756b27ef7a819370484
I20260806 15:59:37.671001 1221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:37.687431 1221 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:37.688349 1221 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:37.696728 1221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7: Generated new TSK 0
I20260806 15:59:37.697463 1221 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.129.keytab.
Entry for principal oryx/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.129.keytab.
I20260806 15:59:37.731424 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:36251
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.129.keytab
--principal=oryx/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:33221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:37.845899 1233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:37.846128 1233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:37.846171 1233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:37.850087 1233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:37.850140 1233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:37.850203 1233 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:59:37.850271 1233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:37.855326 1233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:33221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.129.keytab
--principal=oryx/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.31.190:36251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:37.856245 1233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:37.857203 1233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:37.866510 1239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:37.866537 1241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:37.866688 1233 server_base.cc:1034] running on GCE node
W20260806 15:59:37.866595 1238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:37.867149 1233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:37.867748 1233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:37.868901 1233 hybrid_clock.cc:648] HybridClock initialized: now 1786031977868883 us; error 31 us; skew 500 ppm
Aug 06 15:59:37 dist-test-slave-tm1g krb5kdc[1112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031977, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:37.871580 1233 init.cc:377] Logged in from keytab as oryx/127.25.31.129@KRBTEST.COM (short username oryx)
I20260806 15:59:37.872385 1233 webserver.cc:460] Webserver started at http://127.25.31.129:45089/ using document root <none> and password file <none>
I20260806 15:59:37.872910 1233 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:37.872972 1233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:37.873206 1233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:37.875193 1233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "83923868163148a8a7f4b39ae920f86e"
format_stamp: "Formatted at 2026-08-06 15:59:37 on dist-test-slave-tm1g"
server_key: "c6fbcbfb9816fc0963167540d1c59cd2"
server_key_iv: "b16cfb4ecd2d7dd583b0a23fe25b28ce"
server_key_version: "encryptionkey@0"
I20260806 15:59:37.875757 1233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "83923868163148a8a7f4b39ae920f86e"
format_stamp: "Formatted at 2026-08-06 15:59:37 on dist-test-slave-tm1g"
server_key: "c6fbcbfb9816fc0963167540d1c59cd2"
server_key_iv: "b16cfb4ecd2d7dd583b0a23fe25b28ce"
server_key_version: "encryptionkey@0"
I20260806 15:59:37.879743 1233 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 15:59:37.882268 1250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:37.883463 1233 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:37.883590 1233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "83923868163148a8a7f4b39ae920f86e"
format_stamp: "Formatted at 2026-08-06 15:59:37 on dist-test-slave-tm1g"
server_key: "c6fbcbfb9816fc0963167540d1c59cd2"
server_key_iv: "b16cfb4ecd2d7dd583b0a23fe25b28ce"
server_key_version: "encryptionkey@0"
I20260806 15:59:37.883689 1233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:37.902999 1233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:37.905821 1233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:37.906016 1233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:37.907562 1233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:37.907629 1233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:37.907701 1233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:37.907740 1233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 15:59:37 dist-test-slave-tm1g krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031977, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
I20260806 15:59:37.918742 1233 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:41093
I20260806 15:59:37.918881 1371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:41093 every 8 connection(s)
I20260806 15:59:37.919842 1233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:37.921475 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:37.908352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:53043 (local address 127.25.31.190:36251)
0806 15:59:37.908475 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:37.908479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:37.909501 (+ 1022us) server_negotiation.cc:407] Connection header received
0806 15:59:37.910577 (+ 1076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:37.910581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:37.910649 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:37.910711 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:37.911989 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:37.912899 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:37.914181 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:37.914617 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:37.914754 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:37.917719 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:37.917723 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:37.917726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:37.917754 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:37.919388 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:37.919872 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:37.919879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:37.919881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:37.919980 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:37.920345 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:37.920348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:37.920350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:37.920578 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:37.920670 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:37.921233 (+ 563us) server_negotiation.cc:299] Negotiation successful
0806 15:59:37.921357 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 15:59:37.922153 1259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:37.908852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:36251 (local address 127.25.31.129:53043)
0806 15:59:37.909353 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:37.909395 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:37.910305 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:37.911095 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:37.911101 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:37.911401 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:37.911826 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:37.911837 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:37.913041 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:37.913045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:37.914042 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:37.914052 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:37.914796 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:37.914802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:37.915013 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:37.915596 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:37.915612 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:37.917579 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:37.919513 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:37.919518 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:37.919530 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:37.919770 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:37.920071 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:37.920074 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:37.920075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:37.920226 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:37.920723 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:37.920734 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:37.920935 (+ 201us) client_negotiation.cc:769] Sending connection context
0806 15:59:37.921128 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 15:59:37.921387 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":122,"threads_started":1}
I20260806 15:59:37.923302 1372 heartbeater.cc:347] Connected to a master server at 127.25.31.190:36251
I20260806 15:59:37.923626 1372 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:37.924196 1372 heartbeater.cc:511] Master 127.25.31.190:36251 requested a full tablet report, sending...
I20260806 15:59:37.925879 1159 ts_manager.cc:194] Registered new tserver with Master: 83923868163148a8a7f4b39ae920f86e (127.25.31.129:41093)
I20260806 15:59:37.927037 1159 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:53043
I20260806 15:59:37.927536 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 1233
I20260806 15:59:37.927726 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 15:59:37.928097 25726 external_mini_cluster.cc:1428] Setting key ecd1e1d1b23cd623493c5f6afbefb6f8
I20260806 15:59:37.933555 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:37.925938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:41837 (local address 127.25.31.190:36251)
0806 15:59:37.926063 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:37.926067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:37.926564 (+ 497us) server_negotiation.cc:407] Connection header received
0806 15:59:37.926693 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:37.926696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:37.926737 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:37.926817 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:37.927230 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:37.927914 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:37.929042 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:37.929348 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:37.929448 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:37.930621 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:37.930624 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:37.930626 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:37.930646 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:37.931974 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:37.932400 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:37.932408 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:37.932409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:37.932480 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:37.932762 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:37.932767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:37.932768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:37.932929 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:37.933032 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:37.933290 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 15:59:37.933395 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 15:59:37.933558 1378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:37.925837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:36251 (local address 127.25.31.129:41837)
0806 15:59:37.926484 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:37.926495 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:37.926600 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:37.926833 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:37.926835 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:37.927024 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:37.927116 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:37.927121 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:37.928087 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:37.928090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:37.928906 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:37.928915 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:37.929459 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:37.929463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:37.929558 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:37.929995 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:37.930022 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:37.930494 (+ 472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:37.932092 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:37.932097 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:37.932099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:37.932298 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:37.932581 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:37.932585 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:37.932587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:37.932673 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:37.933026 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:37.933029 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:37.933131 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 15:59:37.933235 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 15:59:37.933373 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for oryx/127.25.31.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.31.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.130.keytab.
Entry for principal oryx/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.130.keytab.
I20260806 15:59:37.954818 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:36251
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.130.keytab
--principal=oryx/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:33221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:38.061797 1381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:38.062026 1381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:38.062098 1381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:38.066072 1381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:38.066125 1381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:38.066162 1381 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:59:38.066239 1381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:38.071308 1381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:33221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.130.keytab
--principal=oryx/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.31.190:36251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:38.072243 1381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:38.073246 1381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:38.082365 1386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:38.082472 1387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:38.082484 1381 server_base.cc:1034] running on GCE node
W20260806 15:59:38.082365 1389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:38.083083 1381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:38.083663 1381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:38.084816 1381 hybrid_clock.cc:648] HybridClock initialized: now 1786031978084785 us; error 43 us; skew 500 ppm
Aug 06 15:59:38 dist-test-slave-tm1g krb5kdc[1112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031978, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:38.087620 1381 init.cc:377] Logged in from keytab as oryx/127.25.31.130@KRBTEST.COM (short username oryx)
I20260806 15:59:38.088418 1381 webserver.cc:460] Webserver started at http://127.25.31.130:35319/ using document root <none> and password file <none>
I20260806 15:59:38.088912 1381 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:38.088969 1381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:38.089125 1381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:38.091064 1381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "99fa529514b547538536434e99737926"
format_stamp: "Formatted at 2026-08-06 15:59:38 on dist-test-slave-tm1g"
server_key: "69e6cf85e6d29e511e4789a367dc6f43"
server_key_iv: "919545b972ac6102a22119b8c074f8cd"
server_key_version: "encryptionkey@0"
I20260806 15:59:38.091614 1381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "99fa529514b547538536434e99737926"
format_stamp: "Formatted at 2026-08-06 15:59:38 on dist-test-slave-tm1g"
server_key: "69e6cf85e6d29e511e4789a367dc6f43"
server_key_iv: "919545b972ac6102a22119b8c074f8cd"
server_key_version: "encryptionkey@0"
I20260806 15:59:38.095125 1381 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 15:59:38.097432 1396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:38.098619 1381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 15:59:38.098739 1381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "99fa529514b547538536434e99737926"
format_stamp: "Formatted at 2026-08-06 15:59:38 on dist-test-slave-tm1g"
server_key: "69e6cf85e6d29e511e4789a367dc6f43"
server_key_iv: "919545b972ac6102a22119b8c074f8cd"
server_key_version: "encryptionkey@0"
I20260806 15:59:38.098839 1381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:38.114449 1381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:38.117478 1381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:38.117671 1381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:38.119125 1381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:38.119190 1381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:38.119256 1381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:38.119294 1381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 15:59:38 dist-test-slave-tm1g krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031978, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
I20260806 15:59:38.129884 1381 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:42789
I20260806 15:59:38.130270 1517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:42789 every 8 connection(s)
I20260806 15:59:38.130995 1381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:38.133183 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.119769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:40423 (local address 127.25.31.190:36251)
0806 15:59:38.119878 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.119896 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.120946 (+ 1050us) server_negotiation.cc:407] Connection header received
0806 15:59:38.122037 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.122041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.122121 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.122183 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:38.123622 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.124376 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.126099 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.126515 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.126644 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.129519 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:38.129523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:38.129526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:38.129557 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:38.131288 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:38.131866 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:38.131869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:38.131871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:38.131965 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:38.132244 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:38.132246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:38.132247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:38.132406 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:38.132477 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:38.132951 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 15:59:38.133065 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 15:59:38.133955 1405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.120319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:36251 (local address 127.25.31.130:40423)
0806 15:59:38.120764 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:38.120804 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:38.121771 (+ 967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:38.122599 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:38.122610 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:38.122986 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:38.123424 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.123436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.124510 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.124514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.125984 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.125996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.126688 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.126692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.126875 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.127986 (+ 1111us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:38.128001 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:38.129350 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:38.131418 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:38.131424 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:38.131438 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:38.131761 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:38.132054 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:38.132057 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:38.132058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:38.132182 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:38.132485 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:38.132491 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:38.132714 (+ 223us) client_negotiation.cc:769] Sending connection context
0806 15:59:38.132911 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 15:59:38.133172 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":132,"threads_started":1}
I20260806 15:59:38.135116 1518 heartbeater.cc:347] Connected to a master server at 127.25.31.190:36251
I20260806 15:59:38.135249 1518 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:38.135778 1518 heartbeater.cc:511] Master 127.25.31.190:36251 requested a full tablet report, sending...
I20260806 15:59:38.136866 1160 ts_manager.cc:194] Registered new tserver with Master: 99fa529514b547538536434e99737926 (127.25.31.130:42789)
I20260806 15:59:38.137648 1160 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:40423
I20260806 15:59:38.140609 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 1381
I20260806 15:59:38.140777 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 15:59:38.141175 25726 external_mini_cluster.cc:1428] Setting key 43cce5afccf8b47b346da3894df64569
I20260806 15:59:38.144824 1524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.137336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:36251 (local address 127.25.31.130:53041)
0806 15:59:38.137611 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:38.137622 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:38.137723 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:38.137957 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:38.137960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:38.138152 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:38.138243 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.138248 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.139005 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.139008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.139570 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.139577 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.140179 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.140182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.140282 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.140804 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:38.140815 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:38.141144 (+ 329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:38.143107 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:38.143109 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:38.143111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:38.143321 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:38.143622 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:38.143624 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:38.143626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:38.143702 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:38.144240 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:38.144243 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:38.144341 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 15:59:38.144496 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 15:59:38.144641 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":97,"threads_started":1}
I20260806 15:59:38.144822 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.137407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:53041 (local address 127.25.31.190:36251)
0806 15:59:38.137519 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.137524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.137685 (+ 161us) server_negotiation.cc:407] Connection header received
0806 15:59:38.137810 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.137813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.137856 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.137940 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:38.138388 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.138882 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.139836 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.140065 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.140165 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.141273 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:38.141278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:38.141281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:38.141306 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:38.142988 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:38.143422 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:38.143426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:38.143428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:38.143524 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:38.143783 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:38.143788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:38.143793 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:38.144105 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:38.144227 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:38.144555 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 15:59:38.144671 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
WARNING: no policy specified for oryx/127.25.31.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.31.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.131.keytab.
Entry for principal oryx/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.131.keytab.
I20260806 15:59:38.168562 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:36251
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.131.keytab
--principal=oryx/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:33221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:38.280043 1527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:38.280272 1527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:38.280314 1527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:38.284422 1527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:38.284476 1527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:38.284514 1527 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 15:59:38.284590 1527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:38.289918 1527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:33221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.131.keytab
--principal=oryx/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.31.190:36251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:38.290932 1527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:38.291989 1527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:59:38.301616 1527 server_base.cc:1034] running on GCE node
W20260806 15:59:38.301591 1535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:38.301566 1532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:38.301568 1533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:38.302176 1527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:38.302817 1527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:38.303965 1527 hybrid_clock.cc:648] HybridClock initialized: now 1786031978303935 us; error 44 us; skew 500 ppm
Aug 06 15:59:38 dist-test-slave-tm1g krb5kdc[1112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031978, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:38.306635 1527 init.cc:377] Logged in from keytab as oryx/127.25.31.131@KRBTEST.COM (short username oryx)
I20260806 15:59:38.307507 1527 webserver.cc:460] Webserver started at http://127.25.31.131:44147/ using document root <none> and password file <none>
I20260806 15:59:38.308053 1527 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:38.308115 1527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:38.308315 1527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:38.310287 1527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "14a6c5967ac7461a807120978c9b17ea"
format_stamp: "Formatted at 2026-08-06 15:59:38 on dist-test-slave-tm1g"
server_key: "c3975e27ade978d77bfd678ef7bc953c"
server_key_iv: "4c80413ff8b4b7e82e03e927b9d75573"
server_key_version: "encryptionkey@0"
I20260806 15:59:38.310900 1527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "14a6c5967ac7461a807120978c9b17ea"
format_stamp: "Formatted at 2026-08-06 15:59:38 on dist-test-slave-tm1g"
server_key: "c3975e27ade978d77bfd678ef7bc953c"
server_key_iv: "4c80413ff8b4b7e82e03e927b9d75573"
server_key_version: "encryptionkey@0"
I20260806 15:59:38.314544 1527 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260806 15:59:38.316905 1543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:38.318091 1527 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:38.318231 1527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "14a6c5967ac7461a807120978c9b17ea"
format_stamp: "Formatted at 2026-08-06 15:59:38 on dist-test-slave-tm1g"
server_key: "c3975e27ade978d77bfd678ef7bc953c"
server_key_iv: "4c80413ff8b4b7e82e03e927b9d75573"
server_key_version: "encryptionkey@0"
I20260806 15:59:38.318378 1527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:38.337406 1527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:38.340293 1527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:38.340487 1527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:38.341912 1527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:38.341982 1527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:38.342073 1527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:38.342113 1527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 15:59:38 dist-test-slave-tm1g krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031978, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
I20260806 15:59:38.353288 1527 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:44087
I20260806 15:59:38.353576 1664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:44087 every 8 connection(s)
I20260806 15:59:38.354444 1527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 15:59:38.356493 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.342625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:58807 (local address 127.25.31.190:36251)
0806 15:59:38.342753 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.342757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.343752 (+ 995us) server_negotiation.cc:407] Connection header received
0806 15:59:38.344735 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.344739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.344800 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.344886 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:38.346101 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.346877 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.348781 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.349127 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.349247 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.352452 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:38.352455 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:38.352457 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:38.352483 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:38.354229 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:38.354930 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:38.354937 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:38.354938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:38.355033 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:38.355382 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:38.355385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:38.355387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:38.355615 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:38.355709 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:38.356189 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 15:59:38.356322 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 15:59:38.357221 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.343159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:36251 (local address 127.25.31.131:58807)
0806 15:59:38.343615 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:38.343656 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:38.344535 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:38.345168 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:38.345174 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:38.345500 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:38.345931 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.345944 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.347031 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.347036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.348636 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.348651 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.349311 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.349318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.349562 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.350199 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:38.350219 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:38.352286 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:38.354399 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:38.354408 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:38.354424 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:38.354794 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:38.355139 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:38.355147 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:38.355150 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:38.355274 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:38.355716 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:38.355724 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:38.355955 (+ 231us) client_negotiation.cc:769] Sending connection context
0806 15:59:38.356142 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 15:59:38.356445 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":139,"threads_started":1}
I20260806 15:59:38.358423 1665 heartbeater.cc:347] Connected to a master server at 127.25.31.190:36251
I20260806 15:59:38.358590 1665 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:38.359089 1665 heartbeater.cc:511] Master 127.25.31.190:36251 requested a full tablet report, sending...
I20260806 15:59:38.360196 1159 ts_manager.cc:194] Registered new tserver with Master: 14a6c5967ac7461a807120978c9b17ea (127.25.31.131:44087)
I20260806 15:59:38.360908 1159 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:58807
I20260806 15:59:38.364718 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 1527
I20260806 15:59:38.364847 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 15:59:38.365166 25726 external_mini_cluster.cc:1428] Setting key e9bd740d87c352fd51d74da4dd96bf16
I20260806 15:59:38.367409 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:59:38.368338 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.360524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:51529 (local address 127.25.31.190:36251)
0806 15:59:38.360629 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.360633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.361452 (+ 819us) server_negotiation.cc:407] Connection header received
0806 15:59:38.361521 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.361525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.361573 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.361646 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:38.362249 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.362795 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.363682 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.363909 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.364010 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.365116 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:38.365120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:38.365123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:38.365146 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:38.366776 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:38.367200 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:38.367203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:38.367205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:38.367296 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:38.367590 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:38.367592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:38.367593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:38.367756 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:38.367834 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:38.368139 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 15:59:38.368232 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 15:59:38.368546 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.360456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:36251 (local address 127.25.31.131:51529)
0806 15:59:38.361357 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:38.361377 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:38.361457 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:38.361685 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:38.361689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:38.361921 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:38.362135 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.362142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.362918 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.362922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.363558 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.363566 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.364016 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.364019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.364128 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.364583 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:38.364593 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:38.364967 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:38.366885 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:38.366888 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:38.366889 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:38.367111 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:38.367398 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:38.367401 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:38.367403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:38.367501 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:38.367864 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:38.367867 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:38.367977 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 15:59:38.368092 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 15:59:38.368254 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":28,"thread_start_us":123,"threads_started":1}
Aug 06 15:59:38 dist-test-slave-tm1g krb5kdc[1112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031978, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 15:59:38 dist-test-slave-tm1g krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:38 dist-test-slave-tm1g krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:59:38.391995 1210 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:38.385667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32922 (local address 127.25.31.190:36251)
0806 15:59:38.385791 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.385794 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.385808 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:59:38.385851 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.385854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.385894 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.385965 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:38.386722 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.387336 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.388203 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.388454 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.388531 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.391824 (+ 3293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32922: BlockingRecv error: recv got EOF from 127.0.0.1:32922 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 15:59:38 dist-test-slave-tm1g krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031978, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
I20260806 15:59:38.400599 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.393366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32924 (local address 127.25.31.190:36251)
0806 15:59:38.393475 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.393479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.393625 (+ 146us) server_negotiation.cc:407] Connection header received
0806 15:59:38.393719 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.393721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.393758 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.393830 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:38.394427 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.394902 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.395637 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.395831 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.395902 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.397998 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:38.398000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:38.398003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:38.398024 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:38.399306 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:38.399671 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:38.399673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:38.399674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:38.399744 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:38.399955 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:38.399957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:38.399958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:38.400121 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:38.400221 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:38.400396 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 15:59:38.400484 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 15:59:38.403575 1159 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:59:38.406666 1159 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:38.420620 1688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.414954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:44087 (local address 127.0.0.1:49168)
0806 15:59:38.415454 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:38.415504 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:38.415630 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:38.416645 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:38.416651 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:38.416674 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:38.416813 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.416819 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.417918 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.417923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.419143 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.419152 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.420089 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.420093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.420204 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.420243 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 15:59:38.420358 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 15:59:38.420472 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":96,"threads_started":1}
I20260806 15:59:38.420652 1689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.415178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49168 (local address 127.25.31.131:44087)
0806 15:59:38.416215 (+ 1037us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.416229 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.416253 (+ 24us) server_negotiation.cc:407] Connection header received
0806 15:59:38.416331 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.416336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.416492 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.416652 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:38.416932 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.417809 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.419258 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.419904 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.420066 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.420233 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:38.420408 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 15:59:38.420501 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":934,"thread_start_us":123,"threads_started":1}
I20260806 15:59:38.421572 1692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.415731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40866 (local address 127.25.31.130:42789)
0806 15:59:38.416151 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.416162 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.416187 (+ 25us) server_negotiation.cc:407] Connection header received
0806 15:59:38.416271 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.416276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.416489 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.416973 (+ 484us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:38.417110 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.418293 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.419860 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.420949 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.421060 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.421219 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:38.421331 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 15:59:38.421420 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":102,"threads_started":1}
I20260806 15:59:38.421779 1691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.415671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:41093 (local address 127.0.0.1:34034)
0806 15:59:38.416486 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:38.416499 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:38.416595 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:38.417278 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:38.417281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:38.417300 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:38.417373 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.417379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.418686 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.418689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.419821 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.419830 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.421405 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.421408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.421491 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.421512 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 15:59:38.421579 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 15:59:38.421651 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":95,"threads_started":1}
I20260806 15:59:38.421779 1690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.415330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:42789 (local address 127.0.0.1:40866)
0806 15:59:38.415871 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:38.415884 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:38.415960 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:38.416664 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:38.416670 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:38.416687 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:38.416815 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.416822 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.418456 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.418459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.419717 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.419727 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.421074 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.421078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.421184 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.421207 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 15:59:38.421288 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 15:59:38.421362 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":34,"spinlock_wait_cycles":1408,"thread_start_us":99,"threads_started":1}
I20260806 15:59:38.422363 1693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.416139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34034 (local address 127.25.31.129:41093)
0806 15:59:38.416830 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.416845 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.416869 (+ 24us) server_negotiation.cc:407] Connection header received
0806 15:59:38.416946 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.416950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.417131 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.417284 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:38.417486 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.418569 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.419938 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.421095 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.421192 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.421346 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:38.422081 (+ 735us) server_negotiation.cc:299] Negotiation successful
0806 15:59:38.422177 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":90,"threads_started":1}
I20260806 15:59:38.424013 1597 tablet_service.cc:1467] Processing CreateTablet for tablet 5e851c0358984552852314955b5c9ef6 (DEFAULT_TABLE table=test-table [id=1d86e821920147eaa12e6a69428dc9f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:38.425086 1597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e851c0358984552852314955b5c9ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:38.424793 1451 tablet_service.cc:1467] Processing CreateTablet for tablet 5e851c0358984552852314955b5c9ef6 (DEFAULT_TABLE table=test-table [id=1d86e821920147eaa12e6a69428dc9f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:38.425801 1451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e851c0358984552852314955b5c9ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:38.425527 1303 tablet_service.cc:1467] Processing CreateTablet for tablet 5e851c0358984552852314955b5c9ef6 (DEFAULT_TABLE table=test-table [id=1d86e821920147eaa12e6a69428dc9f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:38.426517 1303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e851c0358984552852314955b5c9ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:38.431505 1694 tablet_bootstrap.cc:492] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea: Bootstrap starting.
I20260806 15:59:38.433158 1695 tablet_bootstrap.cc:492] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e: Bootstrap starting.
I20260806 15:59:38.434644 1696 tablet_bootstrap.cc:492] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926: Bootstrap starting.
I20260806 15:59:38.434777 1694 tablet_bootstrap.cc:654] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:38.435768 1694 log.cc:824] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:38.435812 1695 tablet_bootstrap.cc:654] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:38.436522 1696 tablet_bootstrap.cc:654] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:38.436808 1695 log.cc:824] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:38.437294 1696 log.cc:824] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:38.437935 1694 tablet_bootstrap.cc:492] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea: No bootstrap required, opened a new log
I20260806 15:59:38.438133 1694 ts_tablet_manager.cc:1397] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 15:59:38.438808 1695 tablet_bootstrap.cc:492] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e: No bootstrap required, opened a new log
I20260806 15:59:38.439005 1695 ts_tablet_manager.cc:1397] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 15:59:38.439157 1696 tablet_bootstrap.cc:492] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926: No bootstrap required, opened a new log
I20260806 15:59:38.439324 1696 ts_tablet_manager.cc:1397] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 15:59:38.442054 1696 raft_consensus.cc:348] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } }
I20260806 15:59:38.442282 1696 raft_consensus.cc:374] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:38.442361 1696 raft_consensus.cc:729] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99fa529514b547538536434e99737926, State: Initialized, Role: FOLLOWER
I20260806 15:59:38.442561 1694 raft_consensus.cc:348] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } }
I20260806 15:59:38.442829 1694 raft_consensus.cc:374] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:38.442705 1695 raft_consensus.cc:348] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } }
I20260806 15:59:38.442893 1694 raft_consensus.cc:729] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14a6c5967ac7461a807120978c9b17ea, State: Initialized, Role: FOLLOWER
I20260806 15:59:38.442912 1695 raft_consensus.cc:374] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:38.442833 1696 consensus_queue.cc:260] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } }
I20260806 15:59:38.442966 1695 raft_consensus.cc:729] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83923868163148a8a7f4b39ae920f86e, State: Initialized, Role: FOLLOWER
I20260806 15:59:38.443341 1695 consensus_queue.cc:260] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } }
I20260806 15:59:38.443318 1694 consensus_queue.cc:260] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } }
I20260806 15:59:38.443820 1696 ts_tablet_manager.cc:1428] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260806 15:59:38.444001 1372 heartbeater.cc:503] Master 127.25.31.190:36251 was elected leader, sending a full tablet report...
I20260806 15:59:38.444142 1518 heartbeater.cc:503] Master 127.25.31.190:36251 was elected leader, sending a full tablet report...
I20260806 15:59:38.444288 1695 ts_tablet_manager.cc:1428] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 15:59:38.444813 1665 heartbeater.cc:503] Master 127.25.31.190:36251 was elected leader, sending a full tablet report...
I20260806 15:59:38.445014 1694 ts_tablet_manager.cc:1428] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 15:59:38.448511 1703 raft_consensus.cc:482] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:38.448669 1703 raft_consensus.cc:504] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } }
I20260806 15:59:38.449697 1703 leader_election.cc:288] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83923868163148a8a7f4b39ae920f86e (127.25.31.129:41093), 99fa529514b547538536434e99737926 (127.25.31.130:42789)
I20260806 15:59:38.454593 1692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.450215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:34541 (local address 127.25.31.130:42789)
0806 15:59:38.450559 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.450564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.450579 (+ 15us) server_negotiation.cc:407] Connection header received
0806 15:59:38.450655 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.450658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.450696 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.450765 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:38.451029 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.451836 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.453489 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.453967 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.454070 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.454109 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:38.454365 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 15:59:38.454452 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255}
I20260806 15:59:38.454615 1704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.449951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:42789 (local address 127.25.31.131:34541)
0806 15:59:38.450484 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:38.450498 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:38.450578 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:38.450786 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:38.450789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:38.450801 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:38.450909 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.450918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.451961 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.451965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.453359 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.453370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.454082 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.454086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.454188 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.454222 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 15:59:38.454352 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 15:59:38.454452 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":128,"thread_start_us":96,"threads_started":1}
I20260806 15:59:38.454787 1693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.450084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:59283 (local address 127.25.31.129:41093)
0806 15:59:38.450383 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.450388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.450403 (+ 15us) server_negotiation.cc:407] Connection header received
0806 15:59:38.450447 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.450450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.450508 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.450580 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:38.451001 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.451836 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.453690 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.454188 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.454362 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.454409 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:38.454591 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 15:59:38.454676 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260806 15:59:38.454805 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.449951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:41093 (local address 127.25.31.131:59283)
0806 15:59:38.450051 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:38.450069 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:38.450153 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:38.450701 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:38.450703 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:38.450724 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:38.450869 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.450888 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.451961 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.451965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.453558 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.453569 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.454344 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.454348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.454459 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.454482 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 15:59:38.454574 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 15:59:38.454659 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":5}
I20260806 15:59:38.455435 1471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e851c0358984552852314955b5c9ef6" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99fa529514b547538536434e99737926" is_pre_election: true
I20260806 15:59:38.455415 1325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e851c0358984552852314955b5c9ef6" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83923868163148a8a7f4b39ae920f86e" is_pre_election: true
I20260806 15:59:38.455705 1325 raft_consensus.cc:2454] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 0.
I20260806 15:59:38.455705 1471 raft_consensus.cc:2454] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 0.
I20260806 15:59:38.456157 1544 leader_election.cc:302] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 14a6c5967ac7461a807120978c9b17ea, 83923868163148a8a7f4b39ae920f86e, 99fa529514b547538536434e99737926; no voters:
I20260806 15:59:38.456413 1703 raft_consensus.cc:2781] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:38.456506 1703 raft_consensus.cc:482] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:38.456571 1703 raft_consensus.cc:3037] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:38.457617 1703 raft_consensus.cc:504] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } }
I20260806 15:59:38.457940 1703 leader_election.cc:288] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 election: Requested vote from peers 83923868163148a8a7f4b39ae920f86e (127.25.31.129:41093), 99fa529514b547538536434e99737926 (127.25.31.130:42789)
I20260806 15:59:38.458284 1325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e851c0358984552852314955b5c9ef6" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83923868163148a8a7f4b39ae920f86e"
I20260806 15:59:38.458279 1471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e851c0358984552852314955b5c9ef6" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99fa529514b547538536434e99737926"
I20260806 15:59:38.458427 1325 raft_consensus.cc:3037] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:38.458441 1471 raft_consensus.cc:3037] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:38.459707 1325 raft_consensus.cc:2454] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 1.
I20260806 15:59:38.459707 1471 raft_consensus.cc:2454] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 1.
I20260806 15:59:38.460008 1544 leader_election.cc:302] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14a6c5967ac7461a807120978c9b17ea, 83923868163148a8a7f4b39ae920f86e; no voters:
I20260806 15:59:38.460207 1703 raft_consensus.cc:2781] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:38.460399 1703 raft_consensus.cc:686] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [term 1 LEADER]: Becoming Leader. State: Replica: 14a6c5967ac7461a807120978c9b17ea, State: Running, Role: LEADER
I20260806 15:59:38.460701 1703 consensus_queue.cc:237] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } }
I20260806 15:59:38.463965 1159 catalog_manager.cc:5495] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 14a6c5967ac7461a807120978c9b17ea (127.25.31.131). New cstate: current_term: 1 leader_uuid: "14a6c5967ac7461a807120978c9b17ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } health_report { overall_health: UNKNOWN } } }
I20260806 15:59:38.479171 1689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.475972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49184 (local address 127.25.31.131:44087)
0806 15:59:38.476074 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.476078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.476115 (+ 37us) server_negotiation.cc:407] Connection header received
0806 15:59:38.476247 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.476249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.476296 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.476376 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:38.476558 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.477032 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.477810 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.478011 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.478103 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.478432 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:38.478823 (+ 391us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:38.478907 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:38.478968 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 15:59:38.479015 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 15:59:38.484548 1325 raft_consensus.cc:1261] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Refusing update from remote peer 14a6c5967ac7461a807120978c9b17ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:38.484557 1471 raft_consensus.cc:1261] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Refusing update from remote peer 14a6c5967ac7461a807120978c9b17ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:38.485211 1705 consensus_queue.cc:1035] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:38.485394 1703 consensus_queue.cc:1035] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:38.493877 1709 mvcc.cc:205] Tried to move back new op lower bound from 7315586983866261504 to 7315586983777890304. Current Snapshot: MvccSnapshot[applied={T|T < 7315586983866261504}]
I20260806 15:59:38.497231 1710 mvcc.cc:205] Tried to move back new op lower bound from 7315586983866261504 to 7315586983777890304. Current Snapshot: MvccSnapshot[applied={T|T < 7315586983866261504 or (T in {7315586983866261504})}]
I20260806 15:59:38.500816 1159 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:38.500967 1159 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:38.503391 1159 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7: Sending DeleteTablet for 3 replicas of tablet 5e851c0358984552852314955b5c9ef6
I20260806 15:59:38.504408 1451 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e851c0358984552852314955b5c9ef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:38 UTC) from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:40866
I20260806 15:59:38.504419 1303 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e851c0358984552852314955b5c9ef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:38 UTC) from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:34034
I20260806 15:59:38.504908 1717 tablet_replica.cc:330] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e: stopping tablet replica
I20260806 15:59:38.504897 1597 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e851c0358984552852314955b5c9ef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:38 UTC) from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:49168
I20260806 15:59:38.504832 1159 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 15:59:38.505224 1717 raft_consensus.cc:2229] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 15:59:38.505414 1159 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:38.505551 1717 raft_consensus.cc:2258] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:38.505720 1719 tablet_replica.cc:330] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea: stopping tablet replica
I20260806 15:59:38.505832 1718 tablet_replica.cc:330] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926: stopping tablet replica
I20260806 15:59:38.506127 1719 raft_consensus.cc:2229] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:59:38.506462 1718 raft_consensus.cc:2229] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:38.506724 1718 raft_consensus.cc:2258] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:38.507634 1717 ts_tablet_manager.cc:1905] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:38.508097 1719 raft_consensus.cc:2258] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:38.509632 1718 ts_tablet_manager.cc:1905] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:38.510524 1717 ts_tablet_manager.cc:1918] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:38.510648 1717 log.cc:1196] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/wal/wals/5e851c0358984552852314955b5c9ef6
I20260806 15:59:38.510958 1719 ts_tablet_manager.cc:1905] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:38.511006 1717 ts_tablet_manager.cc:1939] T 5e851c0358984552852314955b5c9ef6 P 83923868163148a8a7f4b39ae920f86e: Deleting consensus metadata
I20260806 15:59:38.511339 1303 tablet_service.cc:1467] Processing CreateTablet for tablet b56214a0733b4dc6a855206e61f213bb (DEFAULT_TABLE table=test-table [id=80d91b0046804996a2a99d3875b3857d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:38.511619 1303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b56214a0733b4dc6a855206e61f213bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:38.512247 1143 catalog_manager.cc:4841] TS 83923868163148a8a7f4b39ae920f86e (127.25.31.129:41093): tablet 5e851c0358984552852314955b5c9ef6 (table test-table [id=1d86e821920147eaa12e6a69428dc9f2]) successfully deleted
I20260806 15:59:38.512442 1718 ts_tablet_manager.cc:1918] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:38.512544 1718 log.cc:1196] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/5e851c0358984552852314955b5c9ef6
I20260806 15:59:38.512889 1718 ts_tablet_manager.cc:1939] T 5e851c0358984552852314955b5c9ef6 P 99fa529514b547538536434e99737926: Deleting consensus metadata
I20260806 15:59:38.513845 1695 tablet_bootstrap.cc:492] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e: Bootstrap starting.
I20260806 15:59:38.514008 1597 tablet_service.cc:1467] Processing CreateTablet for tablet b56214a0733b4dc6a855206e61f213bb (DEFAULT_TABLE table=test-table [id=80d91b0046804996a2a99d3875b3857d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:38.514241 1597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b56214a0733b4dc6a855206e61f213bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:38.514780 1695 tablet_bootstrap.cc:654] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:38.515384 1719 ts_tablet_manager.cc:1918] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:38.515489 1719 log.cc:1196] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/5e851c0358984552852314955b5c9ef6
I20260806 15:59:38.515799 1719 ts_tablet_manager.cc:1939] T 5e851c0358984552852314955b5c9ef6 P 14a6c5967ac7461a807120978c9b17ea: Deleting consensus metadata
I20260806 15:59:38.516067 1695 tablet_bootstrap.cc:492] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e: No bootstrap required, opened a new log
I20260806 15:59:38.516144 1695 ts_tablet_manager.cc:1397] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 15:59:38.516508 1694 tablet_bootstrap.cc:492] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea: Bootstrap starting.
I20260806 15:59:38.516731 1144 catalog_manager.cc:4841] TS 99fa529514b547538536434e99737926 (127.25.31.130:42789): tablet 5e851c0358984552852314955b5c9ef6 (table test-table [id=1d86e821920147eaa12e6a69428dc9f2]) successfully deleted
I20260806 15:59:38.516781 1695 raft_consensus.cc:348] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.516893 1695 raft_consensus.cc:374] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:38.516929 1695 raft_consensus.cc:729] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83923868163148a8a7f4b39ae920f86e, State: Initialized, Role: FOLLOWER
I20260806 15:59:38.517055 1695 consensus_queue.cc:260] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.517213 1451 tablet_service.cc:1467] Processing CreateTablet for tablet b56214a0733b4dc6a855206e61f213bb (DEFAULT_TABLE table=test-table [id=80d91b0046804996a2a99d3875b3857d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:38.517346 1143 catalog_manager.cc:4841] TS 14a6c5967ac7461a807120978c9b17ea (127.25.31.131:44087): tablet 5e851c0358984552852314955b5c9ef6 (table test-table [id=1d86e821920147eaa12e6a69428dc9f2]) successfully deleted
I20260806 15:59:38.517419 1451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b56214a0733b4dc6a855206e61f213bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:38.517465 1694 tablet_bootstrap.cc:654] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:38.518751 1694 tablet_bootstrap.cc:492] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea: No bootstrap required, opened a new log
I20260806 15:59:38.518837 1694 ts_tablet_manager.cc:1397] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:38.519356 1694 raft_consensus.cc:348] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.519464 1694 raft_consensus.cc:374] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:38.519508 1694 raft_consensus.cc:729] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14a6c5967ac7461a807120978c9b17ea, State: Initialized, Role: FOLLOWER
I20260806 15:59:38.519536 1696 tablet_bootstrap.cc:492] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926: Bootstrap starting.
I20260806 15:59:38.519636 1694 consensus_queue.cc:260] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.519876 1694 ts_tablet_manager.cc:1428] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 15:59:38.520119 1695 ts_tablet_manager.cc:1428] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260806 15:59:38.520501 1696 tablet_bootstrap.cc:654] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:38.521757 1696 tablet_bootstrap.cc:492] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926: No bootstrap required, opened a new log
I20260806 15:59:38.521854 1696 ts_tablet_manager.cc:1397] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:38.522377 1696 raft_consensus.cc:348] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.522504 1696 raft_consensus.cc:374] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:38.522545 1696 raft_consensus.cc:729] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99fa529514b547538536434e99737926, State: Initialized, Role: FOLLOWER
I20260806 15:59:38.522665 1696 consensus_queue.cc:260] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.522929 1696 ts_tablet_manager.cc:1428] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260806 15:59:38.605820 1666 tablet.cc:2369] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:59:38.632294 1519 tablet.cc:2369] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:59:38.659076 1705 raft_consensus.cc:482] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:38.659225 1705 raft_consensus.cc:504] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.659680 1705 leader_election.cc:288] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83923868163148a8a7f4b39ae920f86e (127.25.31.129:41093), 99fa529514b547538536434e99737926 (127.25.31.130:42789)
I20260806 15:59:38.660221 1471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56214a0733b4dc6a855206e61f213bb" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99fa529514b547538536434e99737926" is_pre_election: true
I20260806 15:59:38.660223 1325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56214a0733b4dc6a855206e61f213bb" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83923868163148a8a7f4b39ae920f86e" is_pre_election: true
I20260806 15:59:38.660396 1325 raft_consensus.cc:2454] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 0.
I20260806 15:59:38.660409 1471 raft_consensus.cc:2454] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 0.
I20260806 15:59:38.660733 1544 leader_election.cc:302] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14a6c5967ac7461a807120978c9b17ea, 83923868163148a8a7f4b39ae920f86e; no voters:
I20260806 15:59:38.660921 1705 raft_consensus.cc:2781] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:38.660990 1705 raft_consensus.cc:482] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:38.661026 1705 raft_consensus.cc:3037] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:38.662250 1705 raft_consensus.cc:504] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.662637 1705 leader_election.cc:288] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 election: Requested vote from peers 83923868163148a8a7f4b39ae920f86e (127.25.31.129:41093), 99fa529514b547538536434e99737926 (127.25.31.130:42789)
I20260806 15:59:38.663028 1325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56214a0733b4dc6a855206e61f213bb" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83923868163148a8a7f4b39ae920f86e"
I20260806 15:59:38.663172 1325 raft_consensus.cc:3037] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:38.664325 1325 raft_consensus.cc:2454] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 1.
I20260806 15:59:38.664588 1544 leader_election.cc:302] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14a6c5967ac7461a807120978c9b17ea, 83923868163148a8a7f4b39ae920f86e; no voters:
I20260806 15:59:38.664769 1705 raft_consensus.cc:2781] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:38.664873 1705 raft_consensus.cc:686] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [term 1 LEADER]: Becoming Leader. State: Replica: 14a6c5967ac7461a807120978c9b17ea, State: Running, Role: LEADER
I20260806 15:59:38.664928 1471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56214a0733b4dc6a855206e61f213bb" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99fa529514b547538536434e99737926"
I20260806 15:59:38.665047 1471 raft_consensus.cc:3037] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:38.665026 1705 consensus_queue.cc:237] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.666131 1471 raft_consensus.cc:2454] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 1.
I20260806 15:59:38.667025 1159 catalog_manager.cc:5495] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 14a6c5967ac7461a807120978c9b17ea (127.25.31.131). New cstate: current_term: 1 leader_uuid: "14a6c5967ac7461a807120978c9b17ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } health_report { overall_health: HEALTHY } } }
W20260806 15:59:38.671162 1373 tablet.cc:2369] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 15:59:38 dist-test-slave-tm1g krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031977, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.190@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
I20260806 15:59:38.736387 1731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.728063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:36251 (local address 127.0.0.1:32932)
0806 15:59:38.728362 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:38.728375 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:38.728512 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:38.728763 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:38.728765 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:38.729188 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:38.729286 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.729294 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.730057 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.730061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.730685 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.730693 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.731122 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.731125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.731203 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.731786 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:38.731799 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:38.733249 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:38.734897 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:38.734907 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:38.734913 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:38.735135 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:38.735456 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:38.735464 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:38.735465 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:38.735564 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:38.735912 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:38.735917 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:38.736023 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 15:59:38.736128 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 15:59:38.736246 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":112,"threads_started":1}
I20260806 15:59:38.736532 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.728098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32932 (local address 127.25.31.190:36251)
0806 15:59:38.728240 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.728247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.728450 (+ 203us) server_negotiation.cc:407] Connection header received
0806 15:59:38.728616 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.728619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.728662 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.728739 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:38.729413 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.729948 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.730798 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.731016 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.731121 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.733403 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:38.733407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:38.733414 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:38.733436 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:38.734769 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:38.735231 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:38.735237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:38.735239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:38.735313 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:38.735651 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:38.735653 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:38.735654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:38.735822 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:38.735898 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:38.736215 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 15:59:38.736313 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 15:59:38.740190 1159 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:32932:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 15:59:38.740924 1159 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:38.745610 1303 tablet_service.cc:1467] Processing CreateTablet for tablet bff7de237b364e6c8b91501d4f66c146 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6008989ac8f6435c861e8862f37e8f0e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:59:38.745743 1451 tablet_service.cc:1467] Processing CreateTablet for tablet bff7de237b364e6c8b91501d4f66c146 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6008989ac8f6435c861e8862f37e8f0e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:59:38.746083 1303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bff7de237b364e6c8b91501d4f66c146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:38.746137 1451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bff7de237b364e6c8b91501d4f66c146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:38.746133 1597 tablet_service.cc:1467] Processing CreateTablet for tablet bff7de237b364e6c8b91501d4f66c146 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6008989ac8f6435c861e8862f37e8f0e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 15:59:38.746517 1597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bff7de237b364e6c8b91501d4f66c146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:38.748620 1695 tablet_bootstrap.cc:492] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e: Bootstrap starting.
I20260806 15:59:38.749156 1696 tablet_bootstrap.cc:492] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926: Bootstrap starting.
I20260806 15:59:38.749181 1694 tablet_bootstrap.cc:492] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea: Bootstrap starting.
I20260806 15:59:38.749645 1695 tablet_bootstrap.cc:654] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:38.749897 1696 tablet_bootstrap.cc:654] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:38.750067 1694 tablet_bootstrap.cc:654] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:38.751243 1695 tablet_bootstrap.cc:492] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e: No bootstrap required, opened a new log
I20260806 15:59:38.751292 1696 tablet_bootstrap.cc:492] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926: No bootstrap required, opened a new log
I20260806 15:59:38.751338 1695 ts_tablet_manager.cc:1397] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 15:59:38.751382 1696 ts_tablet_manager.cc:1397] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 15:59:38.751457 1694 tablet_bootstrap.cc:492] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea: No bootstrap required, opened a new log
I20260806 15:59:38.751542 1694 ts_tablet_manager.cc:1397] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 15:59:38.751817 1696 raft_consensus.cc:348] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.751832 1695 raft_consensus.cc:348] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.751946 1696 raft_consensus.cc:374] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:38.751962 1695 raft_consensus.cc:374] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:38.751910 1694 raft_consensus.cc:348] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.751982 1696 raft_consensus.cc:729] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99fa529514b547538536434e99737926, State: Initialized, Role: FOLLOWER
I20260806 15:59:38.752015 1694 raft_consensus.cc:374] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:38.752005 1695 raft_consensus.cc:729] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83923868163148a8a7f4b39ae920f86e, State: Initialized, Role: FOLLOWER
I20260806 15:59:38.752058 1694 raft_consensus.cc:729] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14a6c5967ac7461a807120978c9b17ea, State: Initialized, Role: FOLLOWER
I20260806 15:59:38.752113 1696 consensus_queue.cc:260] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.752171 1694 consensus_queue.cc:260] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.752180 1695 consensus_queue.cc:260] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.752413 1696 ts_tablet_manager.cc:1428] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 15:59:38.752413 1694 ts_tablet_manager.cc:1428] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 15:59:38.752439 1702 tablet_replica.cc:439] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } }
I20260806 15:59:38.752568 1702 tablet_replica.cc:442] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:38.752548 1701 tablet_replica.cc:439] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } }
I20260806 15:59:38.752677 1701 tablet_replica.cc:442] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:38.752425 1703 tablet_replica.cc:439] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } }
I20260806 15:59:38.753043 1695 ts_tablet_manager.cc:1428] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:59:38.753101 1703 tablet_replica.cc:442] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:38.787600 1703 raft_consensus.cc:482] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:38.787736 1703 raft_consensus.cc:504] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.788142 1703 leader_election.cc:288] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83923868163148a8a7f4b39ae920f86e (127.25.31.129:41093), 99fa529514b547538536434e99737926 (127.25.31.130:42789)
I20260806 15:59:38.788591 1325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bff7de237b364e6c8b91501d4f66c146" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83923868163148a8a7f4b39ae920f86e" is_pre_election: true
I20260806 15:59:38.788616 1471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bff7de237b364e6c8b91501d4f66c146" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99fa529514b547538536434e99737926" is_pre_election: true
I20260806 15:59:38.788729 1325 raft_consensus.cc:2454] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 0.
I20260806 15:59:38.788782 1471 raft_consensus.cc:2454] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 0.
I20260806 15:59:38.789068 1544 leader_election.cc:302] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14a6c5967ac7461a807120978c9b17ea, 83923868163148a8a7f4b39ae920f86e; no voters:
I20260806 15:59:38.789268 1703 raft_consensus.cc:2781] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:38.789325 1703 raft_consensus.cc:482] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:38.789353 1703 raft_consensus.cc:3037] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:38.790426 1703 raft_consensus.cc:504] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.790747 1703 leader_election.cc:288] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 election: Requested vote from peers 83923868163148a8a7f4b39ae920f86e (127.25.31.129:41093), 99fa529514b547538536434e99737926 (127.25.31.130:42789)
I20260806 15:59:38.791069 1325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bff7de237b364e6c8b91501d4f66c146" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83923868163148a8a7f4b39ae920f86e"
I20260806 15:59:38.791117 1471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bff7de237b364e6c8b91501d4f66c146" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99fa529514b547538536434e99737926"
I20260806 15:59:38.791183 1325 raft_consensus.cc:3037] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:38.791267 1471 raft_consensus.cc:3037] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:38.792137 1325 raft_consensus.cc:2454] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 1.
I20260806 15:59:38.792296 1471 raft_consensus.cc:2454] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 1.
I20260806 15:59:38.792414 1544 leader_election.cc:302] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14a6c5967ac7461a807120978c9b17ea, 83923868163148a8a7f4b39ae920f86e; no voters:
I20260806 15:59:38.792697 1703 raft_consensus.cc:2781] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:38.792806 1703 raft_consensus.cc:686] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [term 1 LEADER]: Becoming Leader. State: Replica: 14a6c5967ac7461a807120978c9b17ea, State: Running, Role: LEADER
I20260806 15:59:38.792941 1703 consensus_queue.cc:237] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.793587 1705 tablet_replica.cc:439] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea: TxnStatusTablet state changed. Reason: New leader 14a6c5967ac7461a807120978c9b17ea. Latest consensus state: current_term: 1 leader_uuid: "14a6c5967ac7461a807120978c9b17ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } }
I20260806 15:59:38.793692 1705 tablet_replica.cc:442] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:59:38.793953 1736 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 15:59:38.794167 1736 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 15:59:38.794620 1157 catalog_manager.cc:5495] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 14a6c5967ac7461a807120978c9b17ea (127.25.31.131). New cstate: current_term: 1 leader_uuid: "14a6c5967ac7461a807120978c9b17ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } health_report { overall_health: HEALTHY } } }
I20260806 15:59:38.805441 1689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.802074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49190 (local address 127.25.31.131:44087)
0806 15:59:38.802196 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.802201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.802214 (+ 13us) server_negotiation.cc:407] Connection header received
0806 15:59:38.802409 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.802414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.802470 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.802547 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:38.802829 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.803418 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.804317 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.804552 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.804656 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.804950 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:38.805026 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:38.805101 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:38.805242 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 15:59:38.805308 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 15:59:38.805441 1731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.801986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:44087 (local address 127.0.0.1:49190)
0806 15:59:38.802117 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:38.802136 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:38.802239 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:38.802571 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:38.802574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:38.802582 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:59:38.802717 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.802740 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.803527 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.803531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.804202 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.804211 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.804656 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.804660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.804763 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.804806 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:59:38.805105 (+ 299us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:59:38.805124 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 15:59:38.805208 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 15:59:38.805313 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 15:59:38.807822 1325 raft_consensus.cc:1261] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Refusing update from remote peer 14a6c5967ac7461a807120978c9b17ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:38.807852 1471 raft_consensus.cc:1261] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Refusing update from remote peer 14a6c5967ac7461a807120978c9b17ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:38.808264 1703 consensus_queue.cc:1035] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:38.808434 1705 consensus_queue.cc:1035] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:38.809760 1702 tablet_replica.cc:439] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e: TxnStatusTablet state changed. Reason: New leader 14a6c5967ac7461a807120978c9b17ea. Latest consensus state: current_term: 1 leader_uuid: "14a6c5967ac7461a807120978c9b17ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } }
I20260806 15:59:38.809767 1701 tablet_replica.cc:439] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926: TxnStatusTablet state changed. Reason: New leader 14a6c5967ac7461a807120978c9b17ea. Latest consensus state: current_term: 1 leader_uuid: "14a6c5967ac7461a807120978c9b17ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } }
I20260806 15:59:38.809882 1701 tablet_replica.cc:442] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:38.809906 1702 tablet_replica.cc:442] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:38.812100 1705 tablet_replica.cc:439] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "14a6c5967ac7461a807120978c9b17ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } }
I20260806 15:59:38.812207 1705 tablet_replica.cc:442] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:59:38.812513 1702 tablet_replica.cc:439] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "14a6c5967ac7461a807120978c9b17ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } }
I20260806 15:59:38.812606 1702 tablet_replica.cc:442] T bff7de237b364e6c8b91501d4f66c146 P 83923868163148a8a7f4b39ae920f86e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:38.814108 1703 tablet_replica.cc:439] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "14a6c5967ac7461a807120978c9b17ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } }
I20260806 15:59:38.814213 1703 tablet_replica.cc:442] T bff7de237b364e6c8b91501d4f66c146 P 14a6c5967ac7461a807120978c9b17ea: This TxnStatusTablet replica's current role is: LEADER
I20260806 15:59:38.815852 1701 tablet_replica.cc:439] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "14a6c5967ac7461a807120978c9b17ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } } }
I20260806 15:59:38.815963 1701 tablet_replica.cc:442] T bff7de237b364e6c8b91501d4f66c146 P 99fa529514b547538536434e99737926: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 15:59:38.826448 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.822906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:44087 (local address 127.25.31.131:40087)
0806 15:59:38.823052 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:38.823067 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:38.823168 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:38.823376 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:38.823379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:38.823386 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:59:38.823492 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.823498 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.824241 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.824244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.825215 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:38.825226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.825699 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.825702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:38.825788 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.825838 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:59:38.826100 (+ 262us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:59:38.826106 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 15:59:38.826175 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 15:59:38.826289 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":19}
I20260806 15:59:38.826455 1689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.822948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:40087 (local address 127.25.31.131:44087)
0806 15:59:38.823045 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.823055 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.823114 (+ 59us) server_negotiation.cc:407] Connection header received
0806 15:59:38.823252 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.823254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.823290 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.823348 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:38.823592 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.824145 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.825354 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.825582 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.825683 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.825960 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:38.826022 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:38.826076 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:38.826181 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 15:59:38.826257 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":34}
I20260806 15:59:38.832531 1325 raft_consensus.cc:1261] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Refusing update from remote peer 14a6c5967ac7461a807120978c9b17ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:38.832571 1471 raft_consensus.cc:1261] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Refusing update from remote peer 14a6c5967ac7461a807120978c9b17ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:38.832945 1705 consensus_queue.cc:1035] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:38.833088 1703 consensus_queue.cc:1035] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:38.838858 1708 mvcc.cc:205] Tried to move back new op lower bound from 7315586985294983168 to 7315586984613539840. Current Snapshot: MvccSnapshot[applied={T|T < 7315586985294983168}]
I20260806 15:59:38.870287 1159 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:38.870484 1159 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:38.872927 1159 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7: Sending DeleteTablet for 3 replicas of tablet b56214a0733b4dc6a855206e61f213bb
I20260806 15:59:38.873437 1303 tablet_service.cc:1514] Processing DeleteTablet for tablet b56214a0733b4dc6a855206e61f213bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:38 UTC) from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:34034
I20260806 15:59:38.873564 1596 tablet_service.cc:1514] Processing DeleteTablet for tablet b56214a0733b4dc6a855206e61f213bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:38 UTC) from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:49168
I20260806 15:59:38.873626 1717 tablet_replica.cc:330] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e: stopping tablet replica
I20260806 15:59:38.873718 1719 tablet_replica.cc:330] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea: stopping tablet replica
I20260806 15:59:38.873744 1717 raft_consensus.cc:2229] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:38.873819 1719 raft_consensus.cc:2229] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:59:38.873857 1717 raft_consensus.cc:2258] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:38.874009 1719 raft_consensus.cc:2258] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:38.874567 1451 tablet_service.cc:1514] Processing DeleteTablet for tablet b56214a0733b4dc6a855206e61f213bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:38 UTC) from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:40866
I20260806 15:59:38.874727 1718 tablet_replica.cc:330] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926: stopping tablet replica
I20260806 15:59:38.874822 1718 raft_consensus.cc:2229] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:38.874871 1719 ts_tablet_manager.cc:1905] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:38.874923 1718 raft_consensus.cc:2258] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:38.875613 1717 ts_tablet_manager.cc:1905] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:38.877799 1719 ts_tablet_manager.cc:1918] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:59:38.877887 1719 log.cc:1196] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/b56214a0733b4dc6a855206e61f213bb
I20260806 15:59:38.878170 1719 ts_tablet_manager.cc:1939] T b56214a0733b4dc6a855206e61f213bb P 14a6c5967ac7461a807120978c9b17ea: Deleting consensus metadata
I20260806 15:59:38.878453 1717 ts_tablet_manager.cc:1918] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:59:38.879043 1143 catalog_manager.cc:4841] TS 14a6c5967ac7461a807120978c9b17ea (127.25.31.131:44087): tablet b56214a0733b4dc6a855206e61f213bb (table test-table [id=80d91b0046804996a2a99d3875b3857d]) successfully deleted
I20260806 15:59:38.879102 1718 ts_tablet_manager.cc:1905] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:38.881212 1718 ts_tablet_manager.cc:1918] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:59:38.881276 1718 log.cc:1196] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/b56214a0733b4dc6a855206e61f213bb
I20260806 15:59:38.881515 1718 ts_tablet_manager.cc:1939] T b56214a0733b4dc6a855206e61f213bb P 99fa529514b547538536434e99737926: Deleting consensus metadata
I20260806 15:59:38.878592 1717 log.cc:1196] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/wal/wals/b56214a0733b4dc6a855206e61f213bb
I20260806 15:59:38.882128 1144 catalog_manager.cc:4841] TS 99fa529514b547538536434e99737926 (127.25.31.130:42789): tablet b56214a0733b4dc6a855206e61f213bb (table test-table [id=80d91b0046804996a2a99d3875b3857d]) successfully deleted
I20260806 15:59:38.882237 1717 ts_tablet_manager.cc:1939] T b56214a0733b4dc6a855206e61f213bb P 83923868163148a8a7f4b39ae920f86e: Deleting consensus metadata
I20260806 15:59:38.883522 1143 catalog_manager.cc:4841] TS 83923868163148a8a7f4b39ae920f86e (127.25.31.129:41093): tablet b56214a0733b4dc6a855206e61f213bb (table test-table [id=80d91b0046804996a2a99d3875b3857d]) successfully deleted
Aug 06 15:59:38 dist-test-slave-tm1g krb5kdc[1112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 15:59:38 dist-test-slave-tm1g krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:38 dist-test-slave-tm1g krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:59:38.906512 1210 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:38.899402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32934 (local address 127.25.31.190:36251)
0806 15:59:38.899549 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.899555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.899724 (+ 169us) server_negotiation.cc:407] Connection header received
0806 15:59:38.899854 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.899856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.899907 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.899975 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:38.900840 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.901380 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.902651 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.902931 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.903048 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.906366 (+ 3318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32934: BlockingRecv error: recv got EOF from 127.0.0.1:32934 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 15:59:38 dist-test-slave-tm1g krb5kdc[1112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM
I20260806 15:59:38.916154 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:38.907912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32944 (local address 127.25.31.190:36251)
0806 15:59:38.908027 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:38.908031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:38.908101 (+ 70us) server_negotiation.cc:407] Connection header received
0806 15:59:38.908193 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:38.908196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:38.908241 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:38.908328 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:38.908874 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.909645 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.910437 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:38.910778 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:38.910885 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:38.913087 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:38.913091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:38.913093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:38.913119 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:38.914615 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:38.914995 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:38.914999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:38.915000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:38.915087 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:38.915333 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:38.915336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:38.915338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:38.915539 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:38.915666 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:38.915886 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 15:59:38.915999 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 15:59:38.918820 1159 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:59:38.919309 1159 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:38.925659 1451 tablet_service.cc:1467] Processing CreateTablet for tablet 21134f92cd9c4625be4ca0d13f23fcbe (DEFAULT_TABLE table=test-table [id=d5de80edf65e45278027181b1094cdcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:38.925819 1303 tablet_service.cc:1467] Processing CreateTablet for tablet 21134f92cd9c4625be4ca0d13f23fcbe (DEFAULT_TABLE table=test-table [id=d5de80edf65e45278027181b1094cdcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:38.926003 1451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21134f92cd9c4625be4ca0d13f23fcbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:38.926079 1303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21134f92cd9c4625be4ca0d13f23fcbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:38.928519 1695 tablet_bootstrap.cc:492] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e: Bootstrap starting.
I20260806 15:59:38.929406 1696 tablet_bootstrap.cc:492] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926: Bootstrap starting.
I20260806 15:59:38.929443 1695 tablet_bootstrap.cc:654] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:38.930388 1696 tablet_bootstrap.cc:654] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:38.930378 1596 tablet_service.cc:1467] Processing CreateTablet for tablet 21134f92cd9c4625be4ca0d13f23fcbe (DEFAULT_TABLE table=test-table [id=d5de80edf65e45278027181b1094cdcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:38.930648 1596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21134f92cd9c4625be4ca0d13f23fcbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:38.932711 1696 tablet_bootstrap.cc:492] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926: No bootstrap required, opened a new log
I20260806 15:59:38.932790 1696 ts_tablet_manager.cc:1397] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 15:59:38.932984 1694 tablet_bootstrap.cc:492] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea: Bootstrap starting.
I20260806 15:59:38.933255 1696 raft_consensus.cc:348] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.933370 1696 raft_consensus.cc:374] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:38.933404 1696 raft_consensus.cc:729] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99fa529514b547538536434e99737926, State: Initialized, Role: FOLLOWER
I20260806 15:59:38.933528 1696 consensus_queue.cc:260] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.933808 1694 tablet_bootstrap.cc:654] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:38.933812 1696 ts_tablet_manager.cc:1428] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 15:59:38.935042 1695 tablet_bootstrap.cc:492] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e: No bootstrap required, opened a new log
I20260806 15:59:38.935133 1695 ts_tablet_manager.cc:1397] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260806 15:59:38.935536 1695 raft_consensus.cc:348] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.935658 1695 raft_consensus.cc:374] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:38.935698 1695 raft_consensus.cc:729] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83923868163148a8a7f4b39ae920f86e, State: Initialized, Role: FOLLOWER
I20260806 15:59:38.935835 1695 consensus_queue.cc:260] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.936179 1695 ts_tablet_manager.cc:1428] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 15:59:38.936507 1694 tablet_bootstrap.cc:492] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea: No bootstrap required, opened a new log
I20260806 15:59:38.936576 1694 ts_tablet_manager.cc:1397] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260806 15:59:38.936954 1694 raft_consensus.cc:348] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.937042 1694 raft_consensus.cc:374] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:38.937069 1694 raft_consensus.cc:729] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14a6c5967ac7461a807120978c9b17ea, State: Initialized, Role: FOLLOWER
I20260806 15:59:38.937175 1694 consensus_queue.cc:260] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:38.937417 1694 ts_tablet_manager.cc:1428] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 15:59:39.004958 1703 raft_consensus.cc:482] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:39.005110 1703 raft_consensus.cc:504] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:39.005620 1703 leader_election.cc:288] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83923868163148a8a7f4b39ae920f86e (127.25.31.129:41093), 99fa529514b547538536434e99737926 (127.25.31.130:42789)
I20260806 15:59:39.006158 1470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21134f92cd9c4625be4ca0d13f23fcbe" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99fa529514b547538536434e99737926" is_pre_election: true
I20260806 15:59:39.006276 1325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21134f92cd9c4625be4ca0d13f23fcbe" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83923868163148a8a7f4b39ae920f86e" is_pre_election: true
I20260806 15:59:39.006386 1470 raft_consensus.cc:2454] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 0.
I20260806 15:59:39.006520 1325 raft_consensus.cc:2454] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 0.
I20260806 15:59:39.006811 1545 leader_election.cc:302] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14a6c5967ac7461a807120978c9b17ea, 99fa529514b547538536434e99737926; no voters:
I20260806 15:59:39.007031 1703 raft_consensus.cc:2781] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:39.007103 1703 raft_consensus.cc:482] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:39.007155 1703 raft_consensus.cc:3037] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:39.008271 1703 raft_consensus.cc:504] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:39.008628 1703 leader_election.cc:288] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 election: Requested vote from peers 83923868163148a8a7f4b39ae920f86e (127.25.31.129:41093), 99fa529514b547538536434e99737926 (127.25.31.130:42789)
I20260806 15:59:39.008991 1325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21134f92cd9c4625be4ca0d13f23fcbe" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83923868163148a8a7f4b39ae920f86e"
I20260806 15:59:39.009001 1470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21134f92cd9c4625be4ca0d13f23fcbe" candidate_uuid: "14a6c5967ac7461a807120978c9b17ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99fa529514b547538536434e99737926"
I20260806 15:59:39.009114 1470 raft_consensus.cc:3037] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:39.009150 1325 raft_consensus.cc:3037] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:39.010080 1325 raft_consensus.cc:2454] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 1.
I20260806 15:59:39.010142 1470 raft_consensus.cc:2454] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14a6c5967ac7461a807120978c9b17ea in term 1.
I20260806 15:59:39.010443 1544 leader_election.cc:302] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14a6c5967ac7461a807120978c9b17ea, 83923868163148a8a7f4b39ae920f86e; no voters:
I20260806 15:59:39.010682 1703 raft_consensus.cc:2781] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:39.010754 1703 raft_consensus.cc:686] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [term 1 LEADER]: Becoming Leader. State: Replica: 14a6c5967ac7461a807120978c9b17ea, State: Running, Role: LEADER
I20260806 15:59:39.010922 1703 consensus_queue.cc:237] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:39.012523 1159 catalog_manager.cc:5495] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 14a6c5967ac7461a807120978c9b17ea (127.25.31.131). New cstate: current_term: 1 leader_uuid: "14a6c5967ac7461a807120978c9b17ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } health_report { overall_health: HEALTHY } } }
I20260806 15:59:39.032552 1689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:39.029584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49202 (local address 127.25.31.131:44087)
0806 15:59:39.029696 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:39.029700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:39.029732 (+ 32us) server_negotiation.cc:407] Connection header received
0806 15:59:39.029866 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:39.029868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:39.029911 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:39.029974 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:39.030179 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.030734 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.031601 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.031807 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.031880 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:39.032141 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:39.032204 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:39.032256 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:39.032339 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 15:59:39.032383 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 15:59:39.034478 1325 raft_consensus.cc:1261] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Refusing update from remote peer 14a6c5967ac7461a807120978c9b17ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:39.034493 1470 raft_consensus.cc:1261] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Refusing update from remote peer 14a6c5967ac7461a807120978c9b17ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:39.034895 1705 consensus_queue.cc:1035] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:39.035070 1703 consensus_queue.cc:1035] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:39.043018 1160 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:39.043160 1160 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:39.044947 1160 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7: Sending DeleteTablet for 3 replicas of tablet 21134f92cd9c4625be4ca0d13f23fcbe
I20260806 15:59:39.045493 1303 tablet_service.cc:1514] Processing DeleteTablet for tablet 21134f92cd9c4625be4ca0d13f23fcbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:39 UTC) from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:34034
I20260806 15:59:39.045540 1596 tablet_service.cc:1514] Processing DeleteTablet for tablet 21134f92cd9c4625be4ca0d13f23fcbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:39 UTC) from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:49168
I20260806 15:59:39.045517 1451 tablet_service.cc:1514] Processing DeleteTablet for tablet 21134f92cd9c4625be4ca0d13f23fcbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:39 UTC) from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:40866
I20260806 15:59:39.045683 1719 tablet_replica.cc:330] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea: stopping tablet replica
I20260806 15:59:39.045683 1717 tablet_replica.cc:330] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e: stopping tablet replica
I20260806 15:59:39.045799 1717 raft_consensus.cc:2229] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:39.045805 1719 raft_consensus.cc:2229] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:59:39.045931 1717 raft_consensus.cc:2258] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:39.045977 1719 raft_consensus.cc:2258] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:39.046118 1718 tablet_replica.cc:330] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926: stopping tablet replica
I20260806 15:59:39.046240 1718 raft_consensus.cc:2229] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:39.046389 1718 raft_consensus.cc:2258] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:39.046722 1717 ts_tablet_manager.cc:1905] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:39.046754 1719 ts_tablet_manager.cc:1905] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:39.047210 1160 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 15:59:39.047396 1718 ts_tablet_manager.cc:1905] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 15:59:39.047592 1160 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:59:39.049304 1717 ts_tablet_manager.cc:1918] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:39.049355 1719 ts_tablet_manager.cc:1918] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:39.049387 1717 log.cc:1196] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-0/wal/wals/21134f92cd9c4625be4ca0d13f23fcbe
I20260806 15:59:39.049430 1719 log.cc:1196] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/21134f92cd9c4625be4ca0d13f23fcbe
I20260806 15:59:39.049664 1717 ts_tablet_manager.cc:1939] T 21134f92cd9c4625be4ca0d13f23fcbe P 83923868163148a8a7f4b39ae920f86e: Deleting consensus metadata
I20260806 15:59:39.049721 1719 ts_tablet_manager.cc:1939] T 21134f92cd9c4625be4ca0d13f23fcbe P 14a6c5967ac7461a807120978c9b17ea: Deleting consensus metadata
I20260806 15:59:39.049854 1718 ts_tablet_manager.cc:1918] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:59:39.049944 1718 log.cc:1196] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/21134f92cd9c4625be4ca0d13f23fcbe
I20260806 15:59:39.050185 1718 ts_tablet_manager.cc:1939] T 21134f92cd9c4625be4ca0d13f23fcbe P 99fa529514b547538536434e99737926: Deleting consensus metadata
I20260806 15:59:39.050510 1143 catalog_manager.cc:4841] TS 14a6c5967ac7461a807120978c9b17ea (127.25.31.131:44087): tablet 21134f92cd9c4625be4ca0d13f23fcbe (table test-table [id=d5de80edf65e45278027181b1094cdcb]) successfully deleted
I20260806 15:59:39.050765 1143 catalog_manager.cc:4841] TS 83923868163148a8a7f4b39ae920f86e (127.25.31.129:41093): tablet 21134f92cd9c4625be4ca0d13f23fcbe (table test-table [id=d5de80edf65e45278027181b1094cdcb]) successfully deleted
I20260806 15:59:39.050853 1144 catalog_manager.cc:4841] TS 99fa529514b547538536434e99737926 (127.25.31.130:42789): tablet 21134f92cd9c4625be4ca0d13f23fcbe (table test-table [id=d5de80edf65e45278027181b1094cdcb]) successfully deleted
I20260806 15:59:39.052671 1303 tablet_service.cc:1467] Processing CreateTablet for tablet 6208ca5a3ec14b099df33bb9bb996598 (DEFAULT_TABLE table=test-table [id=3aab83965c594fa6916ee435bad0a0ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:39.052706 1451 tablet_service.cc:1467] Processing CreateTablet for tablet 6208ca5a3ec14b099df33bb9bb996598 (DEFAULT_TABLE table=test-table [id=3aab83965c594fa6916ee435bad0a0ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:39.052814 1596 tablet_service.cc:1467] Processing CreateTablet for tablet 6208ca5a3ec14b099df33bb9bb996598 (DEFAULT_TABLE table=test-table [id=3aab83965c594fa6916ee435bad0a0ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:59:39.052942 1303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6208ca5a3ec14b099df33bb9bb996598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:39.052961 1451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6208ca5a3ec14b099df33bb9bb996598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:39.053052 1596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6208ca5a3ec14b099df33bb9bb996598. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:39.054723 1696 tablet_bootstrap.cc:492] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926: Bootstrap starting.
I20260806 15:59:39.055215 1695 tablet_bootstrap.cc:492] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e: Bootstrap starting.
I20260806 15:59:39.055601 1696 tablet_bootstrap.cc:654] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:39.056082 1695 tablet_bootstrap.cc:654] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:39.056712 1694 tablet_bootstrap.cc:492] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea: Bootstrap starting.
I20260806 15:59:39.056859 1696 tablet_bootstrap.cc:492] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926: No bootstrap required, opened a new log
I20260806 15:59:39.056960 1696 ts_tablet_manager.cc:1397] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 15:59:39.057323 1696 raft_consensus.cc:348] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:39.057415 1696 raft_consensus.cc:374] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:39.057437 1696 raft_consensus.cc:729] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99fa529514b547538536434e99737926, State: Initialized, Role: FOLLOWER
I20260806 15:59:39.057474 1694 tablet_bootstrap.cc:654] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:39.057590 1696 consensus_queue.cc:260] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:39.057878 1696 ts_tablet_manager.cc:1428] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 15:59:39.058425 1695 tablet_bootstrap.cc:492] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e: No bootstrap required, opened a new log
I20260806 15:59:39.058519 1695 ts_tablet_manager.cc:1397] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 15:59:39.058995 1695 raft_consensus.cc:348] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:39.059101 1695 raft_consensus.cc:374] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:39.059141 1695 raft_consensus.cc:729] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83923868163148a8a7f4b39ae920f86e, State: Initialized, Role: FOLLOWER
I20260806 15:59:39.059271 1695 consensus_queue.cc:260] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:39.059546 1695 ts_tablet_manager.cc:1428] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 15:59:39.060060 1694 tablet_bootstrap.cc:492] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea: No bootstrap required, opened a new log
I20260806 15:59:39.060127 1694 ts_tablet_manager.cc:1397] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 15:59:39.060532 1694 raft_consensus.cc:348] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:39.060664 1694 raft_consensus.cc:374] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:39.060705 1694 raft_consensus.cc:729] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14a6c5967ac7461a807120978c9b17ea, State: Initialized, Role: FOLLOWER
I20260806 15:59:39.060843 1694 consensus_queue.cc:260] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:39.061116 1694 ts_tablet_manager.cc:1428] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 15:59:39.167871 1702 raft_consensus.cc:482] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:59:39.168022 1702 raft_consensus.cc:504] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:39.169162 1702 leader_election.cc:288] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99fa529514b547538536434e99737926 (127.25.31.130:42789), 14a6c5967ac7461a807120978c9b17ea (127.25.31.131:44087)
I20260806 15:59:39.173492 1771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:39.169379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:42789 (local address 127.25.31.129:59103)
0806 15:59:39.169762 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:39.169774 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:39.169861 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:39.170182 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:39.170188 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:39.170205 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:39.170397 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:39.170403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.171321 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.171323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:39.172321 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:39.172330 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.173021 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.173025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:39.173125 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:39.173158 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 15:59:39.173243 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 15:59:39.173313 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":97,"spinlock_wait_cycles":2944,"thread_start_us":96,"threads_started":1}
I20260806 15:59:39.173480 1773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:39.169497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:59103 (local address 127.25.31.130:42789)
0806 15:59:39.169810 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:39.169816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:39.169836 (+ 20us) server_negotiation.cc:407] Connection header received
0806 15:59:39.170003 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:39.170015 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:39.170077 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:39.170143 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:39.170511 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.171140 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.172448 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.172872 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.173021 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:39.173073 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:39.173263 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 15:59:39.173337 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":106,"threads_started":1}
I20260806 15:59:39.173543 1689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:39.169582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:51823 (local address 127.25.31.131:44087)
0806 15:59:39.169703 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:39.169708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:39.169917 (+ 209us) server_negotiation.cc:407] Connection header received
0806 15:59:39.170060 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:39.170063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:39.170108 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:39.170198 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:59:39.170570 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.171140 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.172485 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.172937 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.173015 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:39.173054 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:39.173349 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 15:59:39.173433 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 15:59:39.173938 1772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:39.169483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:44087 (local address 127.25.31.129:51823)
0806 15:59:39.169810 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:39.169826 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:39.169956 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:39.170229 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:39.170232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:39.170281 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:59:39.170455 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:39.170465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.171302 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.171305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:39.172332 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:39.172342 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.173132 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.173135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:39.173233 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:39.173248 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 15:59:39.173330 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 15:59:39.173402 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":48,"spinlock_wait_cycles":12288,"thread_start_us":69,"threads_started":1}
I20260806 15:59:39.173944 1470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6208ca5a3ec14b099df33bb9bb996598" candidate_uuid: "83923868163148a8a7f4b39ae920f86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99fa529514b547538536434e99737926" is_pre_election: true
I20260806 15:59:39.174091 1470 raft_consensus.cc:2454] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83923868163148a8a7f4b39ae920f86e in term 0.
I20260806 15:59:39.174532 1252 leader_election.cc:302] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83923868163148a8a7f4b39ae920f86e, 99fa529514b547538536434e99737926; no voters:
I20260806 15:59:39.174584 1618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6208ca5a3ec14b099df33bb9bb996598" candidate_uuid: "83923868163148a8a7f4b39ae920f86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14a6c5967ac7461a807120978c9b17ea" is_pre_election: true
I20260806 15:59:39.174801 1618 raft_consensus.cc:2454] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83923868163148a8a7f4b39ae920f86e in term 0.
I20260806 15:59:39.174824 1702 raft_consensus.cc:2781] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:59:39.174902 1702 raft_consensus.cc:482] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:59:39.174960 1702 raft_consensus.cc:3037] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:39.176239 1702 raft_consensus.cc:504] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:39.176605 1702 leader_election.cc:288] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [CANDIDATE]: Term 1 election: Requested vote from peers 99fa529514b547538536434e99737926 (127.25.31.130:42789), 14a6c5967ac7461a807120978c9b17ea (127.25.31.131:44087)
I20260806 15:59:39.176952 1618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6208ca5a3ec14b099df33bb9bb996598" candidate_uuid: "83923868163148a8a7f4b39ae920f86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14a6c5967ac7461a807120978c9b17ea"
I20260806 15:59:39.176981 1470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6208ca5a3ec14b099df33bb9bb996598" candidate_uuid: "83923868163148a8a7f4b39ae920f86e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99fa529514b547538536434e99737926"
I20260806 15:59:39.177052 1618 raft_consensus.cc:3037] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:39.177111 1470 raft_consensus.cc:3037] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:39.177994 1470 raft_consensus.cc:2454] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83923868163148a8a7f4b39ae920f86e in term 1.
I20260806 15:59:39.178049 1618 raft_consensus.cc:2454] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83923868163148a8a7f4b39ae920f86e in term 1.
I20260806 15:59:39.178293 1252 leader_election.cc:302] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83923868163148a8a7f4b39ae920f86e, 99fa529514b547538536434e99737926; no voters:
I20260806 15:59:39.178517 1702 raft_consensus.cc:2781] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:39.178745 1702 raft_consensus.cc:686] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [term 1 LEADER]: Becoming Leader. State: Replica: 83923868163148a8a7f4b39ae920f86e, State: Running, Role: LEADER
I20260806 15:59:39.178984 1702 consensus_queue.cc:237] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } }
I20260806 15:59:39.181248 1159 catalog_manager.cc:5495] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e reported cstate change: term changed from 0 to 1, leader changed from <none> to 83923868163148a8a7f4b39ae920f86e (127.25.31.129). New cstate: current_term: 1 leader_uuid: "83923868163148a8a7f4b39ae920f86e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83923868163148a8a7f4b39ae920f86e" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 41093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 } health_report { overall_health: UNKNOWN } } }
I20260806 15:59:39.218235 1776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:39.214557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34036 (local address 127.25.31.129:41093)
0806 15:59:39.214863 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:39.214869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:39.214880 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:59:39.214948 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:39.214950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:39.215001 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:39.215129 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:39.215369 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.215927 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.216757 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.217010 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.217118 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:39.217386 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:39.217831 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:39.217963 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:39.218024 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 15:59:39.218076 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":114,"threads_started":1}
I20260806 15:59:39.227927 1689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:39.224725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:44289 (local address 127.25.31.131:44087)
0806 15:59:39.224859 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:39.224864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:39.225011 (+ 147us) server_negotiation.cc:407] Connection header received
0806 15:59:39.225161 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:39.225164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:39.225207 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:39.225283 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:39.225547 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.226027 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.226911 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.227125 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.227230 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:39.227521 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:39.227584 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:39.227658 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:39.227746 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 15:59:39.227791 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 15:59:39.227981 1779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:39.224605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:44087 (local address 127.25.31.129:44289)
0806 15:59:39.224929 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:39.224941 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:39.225050 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:39.225307 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:39.225310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:39.225320 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:59:39.225441 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:39.225447 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.226138 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.226142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:39.226784 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:39.226792 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.227248 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.227252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:39.227345 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:39.227385 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:59:39.227662 (+ 277us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:59:39.227671 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 15:59:39.227736 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 15:59:39.227812 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":96,"threads_started":1}
I20260806 15:59:39.233732 1470 raft_consensus.cc:1261] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Refusing update from remote peer 83923868163148a8a7f4b39ae920f86e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:39.233908 1618 raft_consensus.cc:1261] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea [term 1 FOLLOWER]: Refusing update from remote peer 83923868163148a8a7f4b39ae920f86e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:59:39.234193 1702 consensus_queue.cc:1035] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "99fa529514b547538536434e99737926" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:39.234454 1774 consensus_queue.cc:1035] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [LEADER]: Connected to new peer: Peer: permanent_uuid: "14a6c5967ac7461a807120978c9b17ea" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 44087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:59:39.250950 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:39.247825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:41093 (local address 127.25.31.131:54601)
0806 15:59:39.247971 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:39.247987 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:39.248070 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:39.248346 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:39.248349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:39.248358 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 15:59:39.248465 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:39.248470 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.249172 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.249175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:39.249745 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:39.249753 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.250182 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.250185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:39.250285 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:39.250347 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 15:59:39.250666 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 15:59:39.250673 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 15:59:39.250756 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 15:59:39.250839 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 15:59:39.251308 1776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:39.247892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:54601 (local address 127.25.31.129:41093)
0806 15:59:39.248018 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:39.248024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:39.248038 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:59:39.248187 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:39.248191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:39.248243 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:39.248325 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:59:39.248574 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.249084 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.249841 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:39.250073 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:39.250188 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:39.250471 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:59:39.250561 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:59:39.250635 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:39.251134 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 15:59:39.251187 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 15:59:39.274472 1159 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:39.274646 1159 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:59:39.276757 1159 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50898636f72347dfa45c3045c902dba7: Sending DeleteTablet for 3 replicas of tablet 6208ca5a3ec14b099df33bb9bb996598
I20260806 15:59:39.277292 1303 tablet_service.cc:1514] Processing DeleteTablet for tablet 6208ca5a3ec14b099df33bb9bb996598 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:39 UTC) from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:34034
I20260806 15:59:39.277498 1451 tablet_service.cc:1514] Processing DeleteTablet for tablet 6208ca5a3ec14b099df33bb9bb996598 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:39 UTC) from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:40866
I20260806 15:59:39.277559 1717 tablet_replica.cc:330] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e: stopping tablet replica
I20260806 15:59:39.277676 1718 tablet_replica.cc:330] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926: stopping tablet replica
I20260806 15:59:39.277674 1717 raft_consensus.cc:2229] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:59:39.277765 1718 raft_consensus.cc:2229] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:39.277868 1718 raft_consensus.cc:2258] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:39.277964 1717 raft_consensus.cc:2258] T 6208ca5a3ec14b099df33bb9bb996598 P 83923868163148a8a7f4b39ae920f86e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:39.278147 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 1233
I20260806 15:59:39.278592 1718 ts_tablet_manager.cc:1905] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:39.280771 1596 tablet_service.cc:1514] Processing DeleteTablet for tablet 6208ca5a3ec14b099df33bb9bb996598 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:59:39 UTC) from {username='oryx', principal='oryx/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:49168
I20260806 15:59:39.281009 1719 tablet_replica.cc:330] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea: stopping tablet replica
I20260806 15:59:39.281180 1719 raft_consensus.cc:2229] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:59:39.281317 1719 raft_consensus.cc:2258] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:59:39.281512 1718 ts_tablet_manager.cc:1918] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:59:39.281618 1718 log.cc:1196] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/6208ca5a3ec14b099df33bb9bb996598
I20260806 15:59:39.281920 1718 ts_tablet_manager.cc:1939] T 6208ca5a3ec14b099df33bb9bb996598 P 99fa529514b547538536434e99737926: Deleting consensus metadata
I20260806 15:59:39.282073 1719 ts_tablet_manager.cc:1905] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:59:39.282634 1144 catalog_manager.cc:4841] TS 99fa529514b547538536434e99737926 (127.25.31.130:42789): tablet 6208ca5a3ec14b099df33bb9bb996598 (table test-table [id=3aab83965c594fa6916ee435bad0a0ea]) successfully deleted
I20260806 15:59:39.285233 1719 ts_tablet_manager.cc:1918] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 15:59:39.285336 1719 log.cc:1196] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/6208ca5a3ec14b099df33bb9bb996598
I20260806 15:59:39.285625 1719 ts_tablet_manager.cc:1939] T 6208ca5a3ec14b099df33bb9bb996598 P 14a6c5967ac7461a807120978c9b17ea: Deleting consensus metadata
I20260806 15:59:39.286331 1143 catalog_manager.cc:4841] TS 14a6c5967ac7461a807120978c9b17ea (127.25.31.131:44087): tablet 6208ca5a3ec14b099df33bb9bb996598 (table test-table [id=3aab83965c594fa6916ee435bad0a0ea]) successfully deleted
W20260806 15:59:39.287516 1143 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:41093 (error 108)
I20260806 15:59:39.287699 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 1381
I20260806 15:59:39.289552 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:39.289191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:41093 (local address 127.0.0.1:34048)
0806 15:59:39.289398 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:39.289466 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:41093: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":75,"threads_started":1}
W20260806 15:59:39.289748 1143 catalog_manager.cc:4584] TS 83923868163148a8a7f4b39ae920f86e (127.25.31.129:41093): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6208ca5a3ec14b099df33bb9bb996598: Network error: Client connection negotiation failed: client connection to 127.25.31.129:41093: connect: Connection refused (error 111)
I20260806 15:59:39.294888 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 1527
I20260806 15:59:39.302693 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 1125
2026-08-06T15:59:39Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3014 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:59:39 dist-test-slave-tm1g krb5kdc[1788](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:59:39 dist-test-slave-tm1g krb5kdc[1788](info): set up 2 sockets
Aug 06 15:59:39 dist-test-slave-tm1g krb5kdc[1788](info): commencing operation
krb5kdc: starting...
W20260806 15:59:40.355427 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 15:59:40 dist-test-slave-tm1g krb5kdc[1788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:59:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 15:59:40.472347 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:41389
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:33537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:41389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:40.579375 1801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:40.579610 1801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:40.579648 1801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:40.583585 1801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:40.583637 1801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:40.583657 1801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:40.583673 1801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:40.583688 1801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:40.588693 1801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:33537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:41389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:41389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:40.589599 1801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:40.590523 1801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:40.598646 1806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:40.598646 1809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:40.598685 1807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:40.598829 1801 server_base.cc:1034] running on GCE node
I20260806 15:59:40.599514 1801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:40.600535 1801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:40.601733 1801 hybrid_clock.cc:648] HybridClock initialized: now 1786031980601712 us; error 44 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T15:59:40Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1304 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:59:40 dist-test-slave-tm1g krb5kdc[1814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:59:40 dist-test-slave-tm1g krb5kdc[1814](info): set up 2 sockets
Aug 06 15:59:40 dist-test-slave-tm1g krb5kdc[1814](info): commencing operation
krb5kdc: starting...
W20260806 15:59:41.663403 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 15:59:41 dist-test-slave-tm1g krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:59:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 15:59:41.781728 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:37967
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:37967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:41.889498 1827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:41.889739 1827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:41.889777 1827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:41.893720 1827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:41.893772 1827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:41.893791 1827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:41.893808 1827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:41.893823 1827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:41.898903 1827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:37967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:37967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:41.899911 1827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:41.900820 1827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:41.909067 1832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:41.909184 1827 server_base.cc:1034] running on GCE node
W20260806 15:59:41.909056 1833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:41.909034 1835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:41.909811 1827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:41.910842 1827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:41.912029 1827 hybrid_clock.cc:648] HybridClock initialized: now 1786031981912009 us; error 29 us; skew 500 ppm
Aug 06 15:59:41 dist-test-slave-tm1g krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031981, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:41.917706 1827 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 15:59:41.918659 1827 webserver.cc:460] Webserver started at http://127.25.31.190:39655/ using document root <none> and password file <none>
I20260806 15:59:41.919171 1827 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:41.919229 1827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:41.919394 1827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:41.921334 1827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "9a5de5ce82244bd58e37aecf93449b98"
format_stamp: "Formatted at 2026-08-06 15:59:41 on dist-test-slave-tm1g"
server_key: "0b4c1421270a3f3ae83f2064ae1ac35f"
server_key_iv: "98b63dd5be33d4958b8e3ced3648565b"
server_key_version: "encryptionkey@0"
I20260806 15:59:41.921841 1827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "9a5de5ce82244bd58e37aecf93449b98"
format_stamp: "Formatted at 2026-08-06 15:59:41 on dist-test-slave-tm1g"
server_key: "0b4c1421270a3f3ae83f2064ae1ac35f"
server_key_iv: "98b63dd5be33d4958b8e3ced3648565b"
server_key_version: "encryptionkey@0"
I20260806 15:59:41.925463 1827 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 15:59:41.927779 1842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:41.928965 1827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 15:59:41.929083 1827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "9a5de5ce82244bd58e37aecf93449b98"
format_stamp: "Formatted at 2026-08-06 15:59:41 on dist-test-slave-tm1g"
server_key: "0b4c1421270a3f3ae83f2064ae1ac35f"
server_key_iv: "98b63dd5be33d4958b8e3ced3648565b"
server_key_version: "encryptionkey@0"
I20260806 15:59:41.929167 1827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:41.952996 1827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:41.956033 1827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:41.956188 1827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:41.963150 1827 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:37967
I20260806 15:59:41.963171 1897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:37967 every 8 connection(s)
I20260806 15:59:41.964051 1827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:41.966770 1898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:41.967438 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 1827
I20260806 15:59:41.967573 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:41.967908 25726 external_mini_cluster.cc:1428] Setting key 21663e0b0d201510c2150a4e8430e975
I20260806 15:59:41.972770 1898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98: Bootstrap starting.
I20260806 15:59:41.975035 1898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:41.975785 1898 log.cc:824] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98: Log is configured to *not* fsync() on all Append() calls
Aug 06 15:59:41 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:41.977771 1898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98: No bootstrap required, opened a new log
I20260806 15:59:41.981297 1898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5de5ce82244bd58e37aecf93449b98" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37967 } }
I20260806 15:59:41.981539 1898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:41.981596 1901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:41.969352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38836 (local address 127.25.31.190:37967)
0806 15:59:41.969777 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:41.969791 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:41.969826 (+ 35us) server_negotiation.cc:407] Connection header received
0806 15:59:41.970525 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:41.970543 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:41.970855 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:41.971190 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:41.972007 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:41.973016 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:41.973901 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:41.974445 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:41.974632 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:41.977128 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:41.977133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:41.977145 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:41.977171 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:41.978842 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:41.979286 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:41.979292 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:41.979296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:41.979403 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:41.979696 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:41.979700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:41.979702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:41.980082 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:41.980297 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:41.980584 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 15:59:41.980864 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":135,"threads_started":1}
I20260806 15:59:41.981626 1898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a5de5ce82244bd58e37aecf93449b98, State: Initialized, Role: FOLLOWER
I20260806 15:59:41.982162 1898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5de5ce82244bd58e37aecf93449b98" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37967 } }
I20260806 15:59:41.982344 1898 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:41.982424 1898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:41.982519 1898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:41.983373 1898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5de5ce82244bd58e37aecf93449b98" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37967 } }
I20260806 15:59:41.983754 1898 leader_election.cc:302] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a5de5ce82244bd58e37aecf93449b98; no voters:
I20260806 15:59:41.984097 1898 leader_election.cc:288] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:41.984225 1903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:41.984467 1903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 1 LEADER]: Becoming Leader. State: Replica: 9a5de5ce82244bd58e37aecf93449b98, State: Running, Role: LEADER
I20260806 15:59:41.984820 1903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5de5ce82244bd58e37aecf93449b98" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37967 } }
I20260806 15:59:41.985317 1898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:41.986542 1904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a5de5ce82244bd58e37aecf93449b98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5de5ce82244bd58e37aecf93449b98" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37967 } } }
I20260806 15:59:41.986680 1904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:41.986866 1905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a5de5ce82244bd58e37aecf93449b98. Latest consensus state: current_term: 1 leader_uuid: "9a5de5ce82244bd58e37aecf93449b98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5de5ce82244bd58e37aecf93449b98" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37967 } } }
I20260806 15:59:41.987221 1905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:41.987560 1913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:41.990147 1913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:41.995103 1913 catalog_manager.cc:1347] Generated new cluster ID: 4a0051565edc4550b9f8eebeaec9750c
I20260806 15:59:41.995167 1913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:42.015261 1913 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:42.016312 1913 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:42.027204 1913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98: Generated new TSK 0
I20260806 15:59:42.027872 1913 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 15:59:42.061326 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37967
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:42.170599 1924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:42.170840 1924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:42.170881 1924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:42.174793 1924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:42.174841 1924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:42.174929 1924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:42.180018 1924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:42.181035 1924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:42.182011 1924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:42.191669 1932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:42.191709 1929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:42.191736 1930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:42.191846 1924 server_base.cc:1034] running on GCE node
I20260806 15:59:42.192433 1924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:42.193044 1924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:42.194214 1924 hybrid_clock.cc:648] HybridClock initialized: now 1786031982194198 us; error 28 us; skew 500 ppm
Aug 06 15:59:42 dist-test-slave-tm1g krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031982, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:42.196923 1924 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 15:59:42.197742 1924 webserver.cc:460] Webserver started at http://127.25.31.129:43867/ using document root <none> and password file <none>
I20260806 15:59:42.198288 1924 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:42.198438 1924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:42.198674 1924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:42.200649 1924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "bf7ba6fb75084325b0634fab1a1feb47"
format_stamp: "Formatted at 2026-08-06 15:59:42 on dist-test-slave-tm1g"
server_key: "0d78d4403e369a9ef29871409fdaa7da"
server_key_iv: "fcaddca419112b07d37be348eb88efbd"
server_key_version: "encryptionkey@0"
I20260806 15:59:42.201205 1924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "bf7ba6fb75084325b0634fab1a1feb47"
format_stamp: "Formatted at 2026-08-06 15:59:42 on dist-test-slave-tm1g"
server_key: "0d78d4403e369a9ef29871409fdaa7da"
server_key_iv: "fcaddca419112b07d37be348eb88efbd"
server_key_version: "encryptionkey@0"
I20260806 15:59:42.204828 1924 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260806 15:59:42.207283 1940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:42.208662 1924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 15:59:42.208791 1924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "bf7ba6fb75084325b0634fab1a1feb47"
format_stamp: "Formatted at 2026-08-06 15:59:42 on dist-test-slave-tm1g"
server_key: "0d78d4403e369a9ef29871409fdaa7da"
server_key_iv: "fcaddca419112b07d37be348eb88efbd"
server_key_version: "encryptionkey@0"
I20260806 15:59:42.208897 1924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:42.257103 1924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:42.260138 1924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:42.260357 1924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:42.260984 1924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:42.261852 1924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:42.261924 1924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:42.261994 1924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:42.262034 1924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:42.271513 1924 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:41501
I20260806 15:59:42.271533 2057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:41501 every 8 connection(s)
I20260806 15:59:42.272436 1924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:42.277715 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 1924
I20260806 15:59:42.277866 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 15:59:42.278174 25726 external_mini_cluster.cc:1428] Setting key 2752fe6a141cb0b4d8b25b6ab5f08df0
Aug 06 15:59:42 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031982, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:42.288111 1901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:42.274243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:39283 (local address 127.25.31.190:37967)
0806 15:59:42.274446 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:42.274450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:42.275320 (+ 870us) server_negotiation.cc:407] Connection header received
0806 15:59:42.276825 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:42.276829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:42.276891 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:42.276999 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:42.278390 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:42.278957 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:42.280141 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:42.280389 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:42.280497 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:42.283685 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:42.283688 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:42.283690 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:42.283714 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:42.285493 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:42.286145 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:42.286151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:42.286154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:42.286256 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:42.286750 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:42.286753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:42.286755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:42.287022 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:42.287152 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:42.287742 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 15:59:42.287901 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 15:59:42.288825 2060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:42.274659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:39283)
0806 15:59:42.275141 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:42.275186 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:42.276595 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:42.277183 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:42.277191 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:42.277580 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:42.278178 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:42.278194 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:42.279102 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:42.279105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:42.280005 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:42.280015 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:42.280561 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:42.280565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:42.280735 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:42.281397 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:42.281425 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:42.283503 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:42.285619 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:42.285624 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:42.285637 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:42.285951 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:42.286403 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:42.286409 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:42.286413 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:42.286630 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:42.287198 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:42.287204 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:42.287472 (+ 268us) client_negotiation.cc:769] Sending connection context
0806 15:59:42.287635 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 15:59:42.287918 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
I20260806 15:59:42.290203 2058 heartbeater.cc:347] Connected to a master server at 127.25.31.190:37967
I20260806 15:59:42.290481 2058 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:42.290988 2058 heartbeater.cc:511] Master 127.25.31.190:37967 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
I20260806 15:59:42.292680 1860 ts_manager.cc:194] Registered new tserver with Master: bf7ba6fb75084325b0634fab1a1feb47 (127.25.31.129:41501)
I20260806 15:59:42.294224 1860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:39283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 15:59:42.307454 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37967
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:42.416332 2063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:42.416565 2063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:42.416606 2063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:42.420481 2063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:42.420531 2063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:42.420583 2063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:42.425473 2063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:42.426381 2063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:42.427335 2063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:42.436697 2072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:42.436717 2063 server_base.cc:1034] running on GCE node
W20260806 15:59:42.436707 2070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:42.436735 2069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:42.437305 2063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:42.437844 2063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:42.439002 2063 hybrid_clock.cc:648] HybridClock initialized: now 1786031982438983 us; error 29 us; skew 500 ppm
Aug 06 15:59:42 dist-test-slave-tm1g krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031982, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:42.441661 2063 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 15:59:42.442662 2063 webserver.cc:460] Webserver started at http://127.25.31.130:33599/ using document root <none> and password file <none>
I20260806 15:59:42.443192 2063 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:42.443250 2063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:42.443478 2063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:42.445405 2063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "928daea512814d9280725e303c5397ee"
format_stamp: "Formatted at 2026-08-06 15:59:42 on dist-test-slave-tm1g"
server_key: "c844b977aeaf288dd99f34f0844954ee"
server_key_iv: "2bc80ad833dc322b124d0522bca38d6e"
server_key_version: "encryptionkey@0"
I20260806 15:59:42.445959 2063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "928daea512814d9280725e303c5397ee"
format_stamp: "Formatted at 2026-08-06 15:59:42 on dist-test-slave-tm1g"
server_key: "c844b977aeaf288dd99f34f0844954ee"
server_key_iv: "2bc80ad833dc322b124d0522bca38d6e"
server_key_version: "encryptionkey@0"
I20260806 15:59:42.449499 2063 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 15:59:42.451835 2079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:42.452976 2063 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 15:59:42.453104 2063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "928daea512814d9280725e303c5397ee"
format_stamp: "Formatted at 2026-08-06 15:59:42 on dist-test-slave-tm1g"
server_key: "c844b977aeaf288dd99f34f0844954ee"
server_key_iv: "2bc80ad833dc322b124d0522bca38d6e"
server_key_version: "encryptionkey@0"
I20260806 15:59:42.453210 2063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:42.473011 2063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:42.476171 2063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:42.476368 2063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:42.476955 2063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:42.477748 2063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:42.477824 2063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:42.477896 2063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:42.477939 2063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:42.487193 2063 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:36247
I20260806 15:59:42.487216 2195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:36247 every 8 connection(s)
I20260806 15:59:42.488181 2063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:42.493803 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 2063
I20260806 15:59:42.493919 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 15:59:42.494181 25726 external_mini_cluster.cc:1428] Setting key e26e935d848502a7f3b51edaae637ec4
Aug 06 15:59:42 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031982, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:42.501900 1901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:42.489941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:33189 (local address 127.25.31.190:37967)
0806 15:59:42.490084 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:42.490088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:42.490901 (+ 813us) server_negotiation.cc:407] Connection header received
0806 15:59:42.492256 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:42.492259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:42.492316 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:42.492432 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:42.493563 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:42.494243 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:42.495607 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:42.495857 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:42.495969 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:42.498377 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:42.498381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:42.498383 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:42.498406 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:42.499849 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:42.500371 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:42.500375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:42.500377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:42.500462 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:42.500804 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:42.500808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:42.500809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:42.500999 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:42.501110 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:42.501605 (+ 495us) server_negotiation.cc:299] Negotiation successful
0806 15:59:42.501729 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 15:59:42.502732 2198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:42.490279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:33189)
0806 15:59:42.490759 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:42.490801 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:42.492051 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:42.492591 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:42.492599 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:42.492966 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:42.493419 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:42.493432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:42.494426 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:42.494429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:42.495491 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:42.495501 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:42.496008 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:42.496012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:42.496176 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:42.496775 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:42.496791 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:42.498187 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:42.499982 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:42.499988 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:42.500002 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:42.500263 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:42.500557 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:42.500561 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:42.500563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:42.500706 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:42.501109 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:42.501115 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:42.501351 (+ 236us) client_negotiation.cc:769] Sending connection context
0806 15:59:42.501545 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 15:59:42.501858 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":107,"threads_started":1}
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
I20260806 15:59:42.503971 2196 heartbeater.cc:347] Connected to a master server at 127.25.31.190:37967
I20260806 15:59:42.504179 2196 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:42.504654 2196 heartbeater.cc:511] Master 127.25.31.190:37967 requested a full tablet report, sending...
I20260806 15:59:42.505666 1860 ts_manager.cc:194] Registered new tserver with Master: 928daea512814d9280725e303c5397ee (127.25.31.130:36247)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
I20260806 15:59:42.506464 1860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:33189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 15:59:42.521399 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37967
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:42.631022 2201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:42.631253 2201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:42.631294 2201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:42.635416 2201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:42.635464 2201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:42.635550 2201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:42.640555 2201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:42.641605 2201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:42.642584 2201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:42.651522 2208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:42.651522 2210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:42.651633 2207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:42.651867 2201 server_base.cc:1034] running on GCE node
I20260806 15:59:42.652297 2201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:42.652956 2201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:42.654114 2201 hybrid_clock.cc:648] HybridClock initialized: now 1786031982654085 us; error 41 us; skew 500 ppm
Aug 06 15:59:42 dist-test-slave-tm1g krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031982, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:42.656963 2201 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 15:59:42.657763 2201 webserver.cc:460] Webserver started at http://127.25.31.131:33499/ using document root <none> and password file <none>
I20260806 15:59:42.658286 2201 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:42.658406 2201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:42.658627 2201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:42.660554 2201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "a6d9dad2bc48478abd878ab1806c8eea"
format_stamp: "Formatted at 2026-08-06 15:59:42 on dist-test-slave-tm1g"
server_key: "2d1d29d93ba446c1a96c1c747fac3294"
server_key_iv: "dd926824eaabd94496df02a380e463cc"
server_key_version: "encryptionkey@0"
I20260806 15:59:42.661098 2201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "a6d9dad2bc48478abd878ab1806c8eea"
format_stamp: "Formatted at 2026-08-06 15:59:42 on dist-test-slave-tm1g"
server_key: "2d1d29d93ba446c1a96c1c747fac3294"
server_key_iv: "dd926824eaabd94496df02a380e463cc"
server_key_version: "encryptionkey@0"
I20260806 15:59:42.664604 2201 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 15:59:42.666926 2217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:42.668017 2201 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:42.668146 2201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "a6d9dad2bc48478abd878ab1806c8eea"
format_stamp: "Formatted at 2026-08-06 15:59:42 on dist-test-slave-tm1g"
server_key: "2d1d29d93ba446c1a96c1c747fac3294"
server_key_iv: "dd926824eaabd94496df02a380e463cc"
server_key_version: "encryptionkey@0"
I20260806 15:59:42.668251 2201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:42.684602 2201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:42.688045 2201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:42.688248 2201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:42.688814 2201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:42.689628 2201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:42.689675 2201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:42.689745 2201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:42.689788 2201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:42.697821 2201 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:36033
I20260806 15:59:42.697881 2336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:36033 every 8 connection(s)
I20260806 15:59:42.698781 2201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 15:59:42.707573 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 2201
I20260806 15:59:42.707705 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
Aug 06 15:59:42 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031982, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 15:59:42.707995 25726 external_mini_cluster.cc:1428] Setting key 073703f3118e6ceb8346365e558618be
I20260806 15:59:42.712010 1901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:42.700520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:33765 (local address 127.25.31.190:37967)
0806 15:59:42.700675 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:42.700678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:42.701499 (+ 821us) server_negotiation.cc:407] Connection header received
0806 15:59:42.702862 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:42.702865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:42.702918 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:42.702995 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:42.704065 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:42.704590 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:42.705725 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:42.705941 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:42.706016 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:42.708719 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:42.708724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:42.708727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:42.708752 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:42.710270 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:42.710750 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:42.710752 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:42.710754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:42.710827 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:42.711119 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:42.711121 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:42.711122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:42.711287 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:42.711363 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:42.711777 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 15:59:42.711892 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 15:59:42.712778 2339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:42.700923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:33765)
0806 15:59:42.701346 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:42.701393 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:42.702663 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:42.703151 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:42.703157 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:42.703496 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:42.703945 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:42.703957 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:42.704722 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:42.704725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:42.705621 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:42.705631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:42.706064 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:42.706067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:42.706207 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:42.706854 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:42.706869 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:42.708508 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:42.710408 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:42.710412 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:42.710423 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:42.710663 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:42.710914 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:59:42.710916 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:59:42.710917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:59:42.711041 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:59:42.711380 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:42.711384 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:42.711586 (+ 202us) client_negotiation.cc:769] Sending connection context
0806 15:59:42.711728 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 15:59:42.711955 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":108,"threads_started":1}
I20260806 15:59:42.713896 2337 heartbeater.cc:347] Connected to a master server at 127.25.31.190:37967
I20260806 15:59:42.714149 2337 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:42.714681 2337 heartbeater.cc:511] Master 127.25.31.190:37967 requested a full tablet report, sending...
I20260806 15:59:42.715711 1860 ts_manager.cc:194] Registered new tserver with Master: a6d9dad2bc48478abd878ab1806c8eea (127.25.31.131:36033)
I20260806 15:59:42.716279 1860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:33765
I20260806 15:59:42.721935 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.129.keytab.
Entry for principal oryx/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.129.keytab.
WARNING: no policy specified for oryx/127.25.31.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.31.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.130.keytab.
Entry for principal oryx/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.130.keytab.
WARNING: no policy specified for oryx/127.25.31.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.31.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.131.keytab.
Entry for principal oryx/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.131.keytab.
I20260806 15:59:42.792721 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 1924
I20260806 15:59:42.798933 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 2063
I20260806 15:59:42.805284 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 2201
I20260806 15:59:42.811169 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 1827
I20260806 15:59:42.818104 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:37967
--webserver_interface=127.25.31.190
--webserver_port=39655
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:37967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:59:42.926004 2346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:42.926245 2346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:42.926323 2346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:42.930486 2346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:42.930542 2346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:42.930562 2346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:42.930620 2346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:42.930656 2346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:42.935789 2346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:37967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:37967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=39655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:42.936782 2346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:42.937786 2346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:42.946151 2351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:42.946208 2352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:42.946379 2346 server_base.cc:1034] running on GCE node
W20260806 15:59:42.946208 2355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:42.947082 2346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:42.948102 2346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:42.949267 2346 hybrid_clock.cc:648] HybridClock initialized: now 1786031982949249 us; error 32 us; skew 500 ppm
Aug 06 15:59:42 dist-test-slave-tm1g krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031982, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:42.951856 2346 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 15:59:42.952649 2346 webserver.cc:460] Webserver started at http://127.25.31.190:39655/ using document root <none> and password file <none>
I20260806 15:59:42.953140 2346 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:42.953209 2346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:42.956893 2346 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 15:59:42.958959 2362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:42.960132 2346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 15:59:42.960259 2346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "9a5de5ce82244bd58e37aecf93449b98"
format_stamp: "Formatted at 2026-08-06 15:59:41 on dist-test-slave-tm1g"
server_key: "0b4c1421270a3f3ae83f2064ae1ac35f"
server_key_iv: "98b63dd5be33d4958b8e3ced3648565b"
server_key_version: "encryptionkey@0"
I20260806 15:59:42.960772 2346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:42.977545 2346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:42.980490 2346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:42.980700 2346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:42.987860 2346 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:37967
I20260806 15:59:42.987921 2415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:37967 every 8 connection(s)
I20260806 15:59:42.988900 2346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:42.992966 2416 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:59:42.994125 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 2346
I20260806 15:59:42.994632 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:41501
--local_ip_for_outbound_sockets=127.25.31.129
--tserver_master_addrs=127.25.31.190:37967
--webserver_port=43867
--webserver_interface=127.25.31.129
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.31.129
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:59:42.994730 2416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98: Bootstrap starting.
I20260806 15:59:43.018002 2416 log.cc:824] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:43.026214 2416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:59:43.026635 2416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98: Bootstrap complete.
I20260806 15:59:43.031030 2416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5de5ce82244bd58e37aecf93449b98" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37967 } }
I20260806 15:59:43.031461 2416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a5de5ce82244bd58e37aecf93449b98, State: Initialized, Role: FOLLOWER
I20260806 15:59:43.032008 2416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5de5ce82244bd58e37aecf93449b98" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37967 } }
I20260806 15:59:43.032124 2416 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:43.032210 2416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:43.032300 2416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 1 FOLLOWER]: Advancing to term 2
I20260806 15:59:43.034153 2416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5de5ce82244bd58e37aecf93449b98" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37967 } }
I20260806 15:59:43.034502 2416 leader_election.cc:302] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a5de5ce82244bd58e37aecf93449b98; no voters:
I20260806 15:59:43.034843 2416 leader_election.cc:288] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 15:59:43.034938 2420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 15:59:43.035120 2420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [term 2 LEADER]: Becoming Leader. State: Replica: 9a5de5ce82244bd58e37aecf93449b98, State: Running, Role: LEADER
I20260806 15:59:43.035396 2420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5de5ce82244bd58e37aecf93449b98" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37967 } }
I20260806 15:59:43.036028 2416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:43.036280 2420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a5de5ce82244bd58e37aecf93449b98. Latest consensus state: current_term: 2 leader_uuid: "9a5de5ce82244bd58e37aecf93449b98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5de5ce82244bd58e37aecf93449b98" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37967 } } }
I20260806 15:59:43.036396 2420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:43.036734 2424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:43.036777 2421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9a5de5ce82244bd58e37aecf93449b98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a5de5ce82244bd58e37aecf93449b98" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37967 } } }
I20260806 15:59:43.037009 2421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:43.039952 2424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:43.040580 2424 catalog_manager.cc:1259] Loaded cluster ID: 4a0051565edc4550b9f8eebeaec9750c
I20260806 15:59:43.040642 2424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:43.042971 2424 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:43.043635 2424 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9a5de5ce82244bd58e37aecf93449b98: Loaded TSK: 0
I20260806 15:59:43.044803 2424 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 15:59:43.117453 2418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:43.117724 2418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:43.117799 2418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:43.121964 2418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:43.122051 2418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:43.122140 2418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:43.127403 2418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.129.keytab
--principal=oryx/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:41501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=43867
--tserver_master_addrs=127.25.31.190:37967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:43.128364 2418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:43.129365 2418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:43.140133 2444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:43.140129 2443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:43.140323 2446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:43.140278 2418 server_base.cc:1034] running on GCE node
I20260806 15:59:43.140780 2418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:43.141458 2418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:43.142637 2418 hybrid_clock.cc:648] HybridClock initialized: now 1786031983142612 us; error 46 us; skew 500 ppm
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031983, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:43.145383 2418 init.cc:377] Logged in from keytab as oryx/127.25.31.129@KRBTEST.COM (short username oryx)
I20260806 15:59:43.146282 2418 webserver.cc:460] Webserver started at http://127.25.31.129:43867/ using document root <none> and password file <none>
I20260806 15:59:43.146878 2418 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:43.147025 2418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:43.150899 2418 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 15:59:43.153096 2453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:43.154423 2418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 15:59:43.154548 2418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "bf7ba6fb75084325b0634fab1a1feb47"
format_stamp: "Formatted at 2026-08-06 15:59:42 on dist-test-slave-tm1g"
server_key: "0d78d4403e369a9ef29871409fdaa7da"
server_key_iv: "fcaddca419112b07d37be348eb88efbd"
server_key_version: "encryptionkey@0"
I20260806 15:59:43.155082 2418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:43.179804 2418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:43.182731 2418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:43.182934 2418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:43.183542 2418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:43.184396 2418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:43.184468 2418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:43.184540 2418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:43.184583 2418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:43.193634 2418 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:41501
I20260806 15:59:43.193665 2568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:41501 every 8 connection(s)
I20260806 15:59:43.194677 2418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:43.202090 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 2418
I20260806 15:59:43.202608 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:36247
--local_ip_for_outbound_sockets=127.25.31.130
--tserver_master_addrs=127.25.31.190:37967
--webserver_port=33599
--webserver_interface=127.25.31.130
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.31.130
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:59:43.208611 2571 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:43.196683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:53519 (local address 127.25.31.190:37967)
0806 15:59:43.197113 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:43.197129 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:43.197318 (+ 189us) server_negotiation.cc:407] Connection header received
0806 15:59:43.198930 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:43.198940 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:43.199242 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:43.199522 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:43.200598 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.201556 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.202934 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.203724 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.203918 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.207945 (+ 4027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:53519: BlockingRecv error: recv got EOF from 127.25.31.129:53519 (error 108)
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":131,"threads_started":1}
I20260806 15:59:43.208731 2572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:43.196691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:53519)
0806 15:59:43.197154 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:43.197206 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:43.198541 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:43.199636 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:43.199642 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:43.199994 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:43.200444 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.200458 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.201712 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.201715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.202783 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.202800 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.203888 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.203890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.204022 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.204706 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:43.204731 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:43.207725 (+ 2994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":146,"threads_started":1}
W20260806 15:59:43.209268 2569 heartbeater.cc:644] Failed to heartbeat to 127.25.31.190:37967 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.31.190:37967: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:43.215547 2572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:43.209773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:43863)
0806 15:59:43.209973 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:43.209989 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:43.210084 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:43.210357 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:43.210362 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:43.210554 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:43.210694 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.210704 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.211407 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.211413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.212438 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.212452 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.212872 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.212875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.212999 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.213456 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:43.213478 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:43.215405 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114}
W20260806 15:59:43.215646 2571 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:43.209837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:43863 (local address 127.25.31.190:37967)
0806 15:59:43.209952 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:43.209955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:43.210051 (+ 96us) server_negotiation.cc:407] Connection header received
0806 15:59:43.210168 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:43.210170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:43.210208 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:43.210274 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:43.210808 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.211296 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.212556 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.212762 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.212834 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.215529 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:43863: BlockingRecv error: recv got EOF from 127.25.31.129:43863 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:43.221261 2572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:43.216191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:48011)
0806 15:59:43.216368 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:43.216381 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:43.216455 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:43.216665 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:43.216668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:43.216783 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:43.216848 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.216852 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.217839 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.217842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.218397 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.218405 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.218938 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.218941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.218995 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.219454 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:43.219466 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:43.221174 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111}
W20260806 15:59:43.221526 2571 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:43.216311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:48011 (local address 127.25.31.190:37967)
0806 15:59:43.216431 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:43.216436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:43.216449 (+ 13us) server_negotiation.cc:407] Connection header received
0806 15:59:43.216531 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:43.216534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:43.216573 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:43.216643 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:43.216952 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.217729 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.218495 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.218832 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.218924 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.221369 (+ 2445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:48011: BlockingRecv error: recv got EOF from 127.25.31.129:48011 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 15:59:43.221675 2569 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 15:59:43.316408 2573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:43.316700 2573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:43.316776 2573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:43.320787 2573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:43.320876 2573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:43.320967 2573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:43.326192 2573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.130.keytab
--principal=oryx/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:36247
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=33599
--tserver_master_addrs=127.25.31.190:37967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:43.327229 2573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:43.328262 2573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:43.337831 2578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:43.337850 2581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:43.337877 2579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:43.338182 2573 server_base.cc:1034] running on GCE node
I20260806 15:59:43.338598 2573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:43.339218 2573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:43.340384 2573 hybrid_clock.cc:648] HybridClock initialized: now 1786031983340374 us; error 28 us; skew 500 ppm
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031983, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:43.343169 2573 init.cc:377] Logged in from keytab as oryx/127.25.31.130@KRBTEST.COM (short username oryx)
I20260806 15:59:43.343958 2573 webserver.cc:460] Webserver started at http://127.25.31.130:33599/ using document root <none> and password file <none>
I20260806 15:59:43.344455 2573 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:43.344522 2573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:43.348259 2573 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 15:59:43.350204 2588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:43.351413 2573 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:43.351541 2573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "928daea512814d9280725e303c5397ee"
format_stamp: "Formatted at 2026-08-06 15:59:42 on dist-test-slave-tm1g"
server_key: "c844b977aeaf288dd99f34f0844954ee"
server_key_iv: "2bc80ad833dc322b124d0522bca38d6e"
server_key_version: "encryptionkey@0"
I20260806 15:59:43.352064 2573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:43.370723 2573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:43.373621 2573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:43.373817 2573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:43.374437 2573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:43.375232 2573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:43.375303 2573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:43.375375 2573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:43.375419 2573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:43.384128 2573 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:36247
I20260806 15:59:43.384171 2704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:36247 every 8 connection(s)
I20260806 15:59:43.385063 2573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:43.389853 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 2573
I20260806 15:59:43.390386 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:36033
--local_ip_for_outbound_sockets=127.25.31.131
--tserver_master_addrs=127.25.31.190:37967
--webserver_port=33499
--webserver_interface=127.25.31.131
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39279
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.31.131
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:59:43.397261 2571 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:43.386840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:47111 (local address 127.25.31.190:37967)
0806 15:59:43.386993 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:43.386998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:43.387797 (+ 799us) server_negotiation.cc:407] Connection header received
0806 15:59:43.389193 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:43.389196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:43.389247 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:43.389320 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:43.390910 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.391458 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.392739 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.393095 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.393186 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.397087 (+ 3901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:47111: BlockingRecv error: recv got EOF from 127.25.31.130:47111 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 15:59:43.397693 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:43.387216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:47111)
0806 15:59:43.387653 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:43.387695 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:43.388998 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:43.389493 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:43.389500 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:43.390015 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:43.390728 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.390751 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.391592 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.391598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.392595 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.392605 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.393266 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.393273 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.393506 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.394155 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:43.394181 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:43.396873 (+ 2692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":107,"threads_started":1}
W20260806 15:59:43.398330 2705 heartbeater.cc:644] Failed to heartbeat to 127.25.31.190:37967 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.31.190:37967: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:43.404166 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:43.398813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:58205)
0806 15:59:43.398938 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:43.398954 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:43.399033 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:43.399331 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:43.399333 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:43.399460 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:43.399549 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.399553 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.400647 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.400650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.401178 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.401185 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.401842 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.401844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.401917 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.402432 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:43.402444 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:43.404080 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 15:59:43.404335 2571 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:43.398915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:58205 (local address 127.25.31.190:37967)
0806 15:59:43.399062 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:43.399069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:43.399086 (+ 17us) server_negotiation.cc:407] Connection header received
0806 15:59:43.399137 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:43.399142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:43.399201 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:43.399293 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:43.399662 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.400540 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.401293 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.401706 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.401859 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.404225 (+ 2366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:58205: BlockingRecv error: recv got EOF from 127.25.31.130:58205 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:43.409732 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:43.404865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:49655)
0806 15:59:43.404955 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:43.404970 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:43.405042 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:43.405235 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:43.405237 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:43.405364 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:43.405437 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.405442 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.406431 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.406433 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.406934 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.406941 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.407478 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.407481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.407539 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.408009 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:43.408021 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:43.409617 (+ 1596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 15:59:43.409899 2571 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:43.404937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:49655 (local address 127.25.31.190:37967)
0806 15:59:43.405038 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:43.405042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:43.405056 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:59:43.405096 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:43.405099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:43.405136 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:43.405214 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:43.405535 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.406322 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.407037 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.407372 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.407466 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.409745 (+ 2279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:49655: BlockingRecv error: recv got EOF from 127.25.31.130:49655 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 15:59:43.410111 2705 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 15:59:43.503425 2708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:43.503691 2708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:43.503767 2708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:43.507787 2708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:43.507861 2708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:43.507953 2708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:43.513048 2708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39279
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/krb5kdc/oryx_127.25.31.131.keytab
--principal=oryx/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:36033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=33499
--tserver_master_addrs=127.25.31.190:37967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:43.514063 2708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:43.515080 2708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:43.524482 2714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:43.524495 2713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:43.524552 2708 server_base.cc:1034] running on GCE node
W20260806 15:59:43.524495 2716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:43.525089 2708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:43.525699 2708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:43.526893 2708 hybrid_clock.cc:648] HybridClock initialized: now 1786031983526858 us; error 44 us; skew 500 ppm
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786031983, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:59:43.529692 2708 init.cc:377] Logged in from keytab as oryx/127.25.31.131@KRBTEST.COM (short username oryx)
I20260806 15:59:43.530576 2708 webserver.cc:460] Webserver started at http://127.25.31.131:33499/ using document root <none> and password file <none>
I20260806 15:59:43.531131 2708 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:43.531231 2708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:43.535041 2708 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:43.537205 2723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:43.538630 2708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 15:59:43.538749 2708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "a6d9dad2bc48478abd878ab1806c8eea"
format_stamp: "Formatted at 2026-08-06 15:59:42 on dist-test-slave-tm1g"
server_key: "2d1d29d93ba446c1a96c1c747fac3294"
server_key_iv: "dd926824eaabd94496df02a380e463cc"
server_key_version: "encryptionkey@0"
I20260806 15:59:43.539258 2708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:43.566066 2708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:43.569334 2708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:43.569540 2708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:43.570137 2708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:43.570962 2708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:43.571012 2708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:43.571082 2708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:43.571125 2708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:43.580112 2708 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:36033
I20260806 15:59:43.580142 2841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:36033 every 8 connection(s)
I20260806 15:59:43.581105 2708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 15:59:43.587599 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 2708
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:59:43.592905 2571 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:43.582855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:46425 (local address 127.25.31.190:37967)
0806 15:59:43.582994 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:43.582997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:43.583729 (+ 732us) server_negotiation.cc:407] Connection header received
0806 15:59:43.584994 (+ 1265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:43.584996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:43.585045 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:43.585118 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:43.586365 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.586917 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.588371 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.588715 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.588952 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.592763 (+ 3811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:46425: BlockingRecv error: recv got EOF from 127.25.31.131:46425 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 15:59:43.593467 2844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:43.583142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:46425)
0806 15:59:43.583566 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:43.583607 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:43.584792 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:43.585357 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:43.585363 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:43.585696 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:43.586149 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.586162 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.587040 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.587042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.588221 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.588230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.589014 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.589018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.589238 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.589870 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:43.589885 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:43.592606 (+ 2721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":115,"threads_started":1}
W20260806 15:59:43.593984 2842 heartbeater.cc:644] Failed to heartbeat to 127.25.31.190:37967 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.31.190:37967: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:43.598591 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:43.588620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38840 (local address 127.25.31.190:37967)
0806 15:59:43.588962 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:43.588967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:43.588980 (+ 13us) server_negotiation.cc:407] Connection header received
0806 15:59:43.589016 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:43.589018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:43.589062 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:43.589156 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:43.590051 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.590812 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.591978 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.592354 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.592465 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.593987 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:43.593993 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:43.594010 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:59:43.594040 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:43.595883 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:43.596358 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:43.596362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:43.596367 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:43.596484 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:59:43.597133 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:59:43.597135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:59:43.597137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:59:43.597586 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:43.597728 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:43.598099 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 15:59:43.598390 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":95,"threads_started":1}
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:43.599639 2844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:43.594470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:55375)
0806 15:59:43.594600 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:43.594615 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:43.594683 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:43.594943 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:43.594945 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:43.595102 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:43.595183 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.595188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.595895 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.595898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.596673 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.596683 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.597079 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.597082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.597149 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.597557 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:43.597567 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:43.599532 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 15:59:43.599833 2571 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:43.594516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:55375 (local address 127.25.31.190:37967)
0806 15:59:43.594626 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:43.594628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:43.594643 (+ 15us) server_negotiation.cc:407] Connection header received
0806 15:59:43.594792 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:43.594795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:43.594834 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:43.594928 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:43.595293 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.595773 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.596783 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.596989 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.597061 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.599705 (+ 2644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:55375: BlockingRecv error: recv got EOF from 127.25.31.131:55375 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:43 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:43.605031 2844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:43.600170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:57755)
0806 15:59:43.600338 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:43.600352 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:43.600418 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:43.600658 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:43.600661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:43.600816 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:43.600991 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.600998 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.601930 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.601932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.602511 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:43.602519 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.602906 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.602909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:43.602966 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.603321 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:43.603335 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:43.604954 (+ 1619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106}
W20260806 15:59:43.605206 2842 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 15:59:43.605377 2571 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:43.600272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:57755 (local address 127.25.31.190:37967)
0806 15:59:43.600416 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:43.600419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:43.600435 (+ 16us) server_negotiation.cc:407] Connection header received
0806 15:59:43.600514 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:43.600517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:43.600552 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:43.600624 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:43.601137 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.601805 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.602611 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:43.602808 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:43.602882 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:43.605065 (+ 2183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:57755: BlockingRecv error: recv got EOF from 127.25.31.131:57755 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 06 15:59:44 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:44 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:44.227849 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:44.222290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:35021)
0806 15:59:44.222621 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:44.222644 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:44.222738 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:44.222964 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:44.222966 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:44.223144 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:44.223256 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:44.223261 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:44.224024 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:44.224027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:44.224655 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:44.224664 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:44.225156 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:44.225158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:44.225236 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:44.225832 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:44.225844 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:44.227727 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":85,"threads_started":1}
W20260806 15:59:44.228029 2848 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:44.222417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:35021 (local address 127.25.31.190:37967)
0806 15:59:44.222691 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:44.222694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:44.222707 (+ 13us) server_negotiation.cc:407] Connection header received
0806 15:59:44.222807 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:44.222809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:44.222850 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:44.222922 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:44.223360 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:44.223906 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:44.224787 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:44.225043 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:44.225133 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:44.227875 (+ 2742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:35021: BlockingRecv error: recv got EOF from 127.25.31.129:35021 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":115,"threads_started":1}
Aug 06 15:59:44 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:44 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:44.416602 2849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:44.410776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:37945)
0806 15:59:44.411117 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:44.411140 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:44.411277 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:44.411554 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:44.411557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:44.411769 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:44.411885 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:44.411892 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:44.412855 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:44.412860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:44.413444 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:44.413453 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:44.413965 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:44.413969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:44.414055 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:44.414560 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:44.414572 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:44.416403 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":88,"threads_started":1}
W20260806 15:59:44.416657 2848 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:44.410926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:37945 (local address 127.25.31.190:37967)
0806 15:59:44.411095 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:44.411099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:44.411217 (+ 118us) server_negotiation.cc:407] Connection header received
0806 15:59:44.411380 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:44.411385 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:44.411437 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:44.411532 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:44.412009 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:44.412744 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:44.413572 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:44.413834 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:44.413950 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:44.416544 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:37945: BlockingRecv error: recv got EOF from 127.25.31.130:37945 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 15:59:44 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:44 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:44.611660 2850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:44.605910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:38887)
0806 15:59:44.606188 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:44.606204 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:44.606369 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:44.606686 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:44.606688 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:44.606957 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:44.607074 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:44.607081 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:44.607898 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:44.607900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:44.608493 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:44.608501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:44.608967 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:44.608969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:44.609048 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:44.609632 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:44.609645 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:44.611543 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":78,"threads_started":1}
W20260806 15:59:44.611876 2848 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:44.605949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:38887 (local address 127.25.31.190:37967)
0806 15:59:44.606111 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:44.606116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:44.606290 (+ 174us) server_negotiation.cc:407] Connection header received
0806 15:59:44.606479 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:44.606482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:44.606539 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:44.606629 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:44.607200 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:44.607793 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:44.608615 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:44.608858 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:44.608936 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:44.611693 (+ 2757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:38887: BlockingRecv error: recv got EOF from 127.25.31.131:38887 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 15:59:45 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:45 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:45.234037 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:45.228629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:47189)
0806 15:59:45.228914 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:45.228928 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:45.229019 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:45.229266 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:45.229268 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:45.229448 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:45.229549 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:45.229554 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:45.230412 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:45.230415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:45.230992 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:45.231002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:45.231503 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:45.231506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:45.231580 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:45.232082 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:45.232093 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:45.233898 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":89,"threads_started":1}
W20260806 15:59:45.234210 2852 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:45.228720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:47189 (local address 127.25.31.190:37967)
0806 15:59:45.228969 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:45.228972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:45.228985 (+ 13us) server_negotiation.cc:407] Connection header received
0806 15:59:45.229105 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:45.229107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:45.229151 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:45.229227 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:45.229647 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:45.230239 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:45.231092 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:45.231377 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:45.231458 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:45.234058 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:47189: BlockingRecv error: recv got EOF from 127.25.31.129:47189 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":64,"threads_started":1}
Aug 06 15:59:45 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:45 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:45.423044 2853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:45.417403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:50507)
0806 15:59:45.417671 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:45.417686 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:45.417784 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:45.418030 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:45.418032 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:45.418211 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:45.418357 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:45.418363 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:45.419097 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:45.419100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:45.419687 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:45.419695 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:45.420300 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:45.420305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:45.420385 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:45.421045 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:45.421062 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:45.422904 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":69,"threads_started":1}
W20260806 15:59:45.423154 2852 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:45.417555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:50507 (local address 127.25.31.190:37967)
0806 15:59:45.417718 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:45.417722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:45.417737 (+ 15us) server_negotiation.cc:407] Connection header received
0806 15:59:45.417872 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:45.417875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:45.417921 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:45.417991 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:45.418460 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:45.418989 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:45.419824 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:45.420169 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:45.420279 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:45.423039 (+ 2760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:50507: BlockingRecv error: recv got EOF from 127.25.31.130:50507 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 15:59:45 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:45 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:45.618191 2854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:45.612524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:60395)
0806 15:59:45.612824 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:45.612837 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:45.612917 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:45.613166 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:45.613169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:45.613352 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:45.613453 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:45.613459 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:45.614228 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:45.614231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:45.614892 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:45.614902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:45.615330 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:45.615333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:45.615441 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:45.616041 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:45.616079 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:45.618049 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":122,"threads_started":1}
W20260806 15:59:45.618399 2852 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:45.612723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:60395 (local address 127.25.31.190:37967)
0806 15:59:45.612859 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:45.612863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:45.612895 (+ 32us) server_negotiation.cc:407] Connection header received
0806 15:59:45.613012 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:45.613015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:45.613063 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:45.613129 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:45.613566 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:45.614108 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:45.615011 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:45.615229 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:45.615341 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:45.618203 (+ 2862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:60395: BlockingRecv error: recv got EOF from 127.25.31.131:60395 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 15:59:46 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:46 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:46.240990 2855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:46.234951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:42895)
0806 15:59:46.235235 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:46.235249 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:46.235340 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:46.235629 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:46.235631 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:46.235814 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:46.235984 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:46.235991 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:46.236935 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:46.236937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:46.237653 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:46.237664 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:46.238090 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:46.238093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:46.238155 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:46.238737 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:46.238749 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:46.240802 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":95,"threads_started":1}
W20260806 15:59:46.241225 2856 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:46.235027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:42895 (local address 127.25.31.190:37967)
0806 15:59:46.235286 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:46.235290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:46.235304 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:59:46.235426 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:46.235428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:46.235472 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:46.235572 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:46.236121 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:46.236829 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:46.237758 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:46.237993 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:46.238073 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:46.240961 (+ 2888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:42895: BlockingRecv error: recv got EOF from 127.25.31.129:42895 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":107,"threads_started":1}
Aug 06 15:59:46 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:46 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:46.429814 2857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:46.423987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:46205)
0806 15:59:46.424354 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:46.424370 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:46.424498 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:46.424819 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:46.424821 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:46.425088 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:46.425195 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:46.425201 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:46.425963 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:46.425966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:46.426611 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:46.426620 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:46.427047 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:46.427050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:46.427111 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:46.427680 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:46.427693 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:46.429666 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":84,"threads_started":1}
W20260806 15:59:46.429940 2856 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:46.424046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:46205 (local address 127.25.31.190:37967)
0806 15:59:46.424179 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:46.424183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:46.424469 (+ 286us) server_negotiation.cc:407] Connection header received
0806 15:59:46.424603 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:46.424607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:46.424671 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:46.424770 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:46.425313 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:46.425858 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:46.426730 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:46.426951 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:46.427032 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:46.429802 (+ 2770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:46205: BlockingRecv error: recv got EOF from 127.25.31.130:46205 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 15:59:46 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:46 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:46.626259 2858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:46.619582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:39969)
0806 15:59:46.620094 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:46.620110 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:46.620282 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:46.620663 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:46.620666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:46.620922 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:46.621069 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:46.621080 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:46.621984 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:46.621990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:46.622979 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:46.622989 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:46.623490 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:46.623493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:46.623590 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:46.624176 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:46.624197 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:46.626083 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":218,"threads_started":1}
W20260806 15:59:46.626362 2856 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:46.619534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:39969 (local address 127.25.31.190:37967)
0806 15:59:46.619700 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:46.619703 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:46.620249 (+ 546us) server_negotiation.cc:407] Connection header received
0806 15:59:46.620438 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:46.620442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:46.620512 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:46.620609 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:46.621196 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:46.621866 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:46.623100 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:46.623363 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:46.623480 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:46.626216 (+ 2736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:39969: BlockingRecv error: recv got EOF from 127.25.31.131:39969 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 15:59:48.071789 2859 sasl_common.cc:323] Time spent creating new SASL oryx client: real 0.826s user 0.000s sys 0.000s
W20260806 15:59:48.071776 2865 sasl_common.cc:323] Time spent creating new SASL oryx client: real 0.441s user 0.000s sys 0.000s
W20260806 15:59:48.071838 2861 sasl_common.cc:323] Time spent creating new SASL oryx client: real 0.638s user 0.000s sys 0.000s
Aug 06 15:59:48 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:48 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:48 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:48 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:48 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:48 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:48.075476 2861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:47.430754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:37503)
0806 15:59:47.431076 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:47.431097 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:47.431223 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:47.431546 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:47.431548 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:47.431735 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:47.431840 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:47.431845 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:47.432594 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:47.432597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:47.433158 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:47.433165 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:47.433608 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:47.433611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:47.433674 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:48.071922 (+638248us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:48.071943 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:48.075269 (+ 3326us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":102,"threads_started":1}
W20260806 15:59:48.075858 2862 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:47.430804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:37503 (local address 127.25.31.190:37967)
0806 15:59:47.431071 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:47.431083 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:47.431159 (+ 76us) server_negotiation.cc:407] Connection header received
0806 15:59:47.431360 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:47.431363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:47.431424 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:47.431517 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:47.431940 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:47.432502 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:47.433263 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:47.433508 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:47.433595 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:48.075721 (+642126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:37503: BlockingRecv error: recv got EOF from 127.25.31.130:37503 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":63,"threads_started":1}
W20260806 15:59:48.076263 2864 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:47.627212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:58057 (local address 127.25.31.190:37967)
0806 15:59:47.627496 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:47.627500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:47.627781 (+ 281us) server_negotiation.cc:407] Connection header received
0806 15:59:47.627827 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:47.627832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:47.627892 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:47.627989 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:47.628443 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:47.629017 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:47.629849 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:47.630144 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:47.630257 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:48.076127 (+445870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:58057: BlockingRecv error: recv got EOF from 127.25.31.131:58057 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":104,"threads_started":1}
I20260806 15:59:48.076602 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:47.627178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:58057)
0806 15:59:47.627530 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:47.627545 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:47.627684 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:47.628002 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:47.628005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:47.628224 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:47.628319 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:47.628325 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:47.629130 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:47.629135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:47.629723 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:47.629732 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:47.630262 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:47.630266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:47.630394 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:48.071912 (+441518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:48.071940 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:48.076414 (+ 4474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":122,"threads_started":1}
I20260806 15:59:48.077091 2859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:47.241954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:34363)
0806 15:59:47.242268 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:47.242283 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:47.242443 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:47.242769 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:47.242772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:47.243001 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:47.243106 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:47.243112 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:47.244026 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:47.244030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:47.244748 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:47.244764 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:47.245246 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:47.245250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:47.245353 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:48.071926 (+826573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:48.071949 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:48.076928 (+ 4979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":85,"threads_started":1}
W20260806 15:59:48.077181 2860 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:47.241997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:34363 (local address 127.25.31.190:37967)
0806 15:59:47.242467 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:47.242476 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:47.242507 (+ 31us) server_negotiation.cc:407] Connection header received
0806 15:59:47.242567 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:47.242571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:47.242626 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:47.242703 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:47.243242 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:47.243910 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:47.244879 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:47.245126 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:47.245232 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:48.077055 (+831823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:34363: BlockingRecv error: recv got EOF from 127.25.31.129:34363 (error 108)
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":93,"threads_started":1}
Aug 06 15:59:49 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:49 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:49 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:49 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:49 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:49 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:49.083958 2866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:49.076643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:40403)
0806 15:59:49.076937 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:49.076950 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:49.077044 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:49.077352 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:49.077356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:49.077598 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:49.077721 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:49.077728 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:49.079045 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:49.079050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:49.079985 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:49.079995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:49.080523 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:49.080527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:49.080620 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:49.081232 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:49.081252 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:49.083785 (+ 2533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":112,"threads_started":1}
W20260806 15:59:49.084040 2867 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:49.076750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:40403 (local address 127.25.31.190:37967)
0806 15:59:49.076995 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:49.077000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:49.077016 (+ 16us) server_negotiation.cc:407] Connection header received
0806 15:59:49.077143 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:49.077149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:49.077208 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:49.077288 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:49.078093 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:49.078942 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:49.080106 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:49.080398 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:49.080488 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:49.083902 (+ 3414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:40403: BlockingRecv error: recv got EOF from 127.25.31.130:40403 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":91,"threads_started":1}
I20260806 15:59:49.084151 2868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:49.077433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:36433)
0806 15:59:49.077772 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:49.077786 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:49.078081 (+ 295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:49.078403 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:49.078407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:49.078606 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:49.078707 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:49.078713 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:49.079550 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:49.079553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:49.080239 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:49.080249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:49.080784 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:49.080788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:49.080891 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:49.081411 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:49.081430 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:49.083994 (+ 2564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":78,"threads_started":1}
W20260806 15:59:49.084407 2869 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:49.077535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:36433 (local address 127.25.31.190:37967)
0806 15:59:49.077755 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:49.077761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:49.077867 (+ 106us) server_negotiation.cc:407] Connection header received
0806 15:59:49.078200 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:49.078203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:49.078256 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:49.078372 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:49.078826 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:49.079438 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:49.080364 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:49.080667 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:49.080776 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:49.084277 (+ 3501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:36433: BlockingRecv error: recv got EOF from 127.25.31.131:36433 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":58,"threads_started":1}
I20260806 15:59:49.084568 2870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:49.077848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:56287)
0806 15:59:49.078085 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:49.078097 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:49.078198 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:49.078668 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:49.078672 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:49.078895 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:49.078998 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:49.079006 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:49.079987 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:49.079991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:49.080840 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:49.080855 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:49.081395 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:49.081400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:49.081494 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:49.081976 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:49.081994 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:49.084365 (+ 2371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":77,"threads_started":1}
W20260806 15:59:49.084805 2871 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:49.077899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:56287 (local address 127.25.31.190:37967)
0806 15:59:49.078461 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:49.078465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:49.078487 (+ 22us) server_negotiation.cc:407] Connection header received
0806 15:59:49.078511 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:49.078513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:49.078541 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:49.078637 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:49.079098 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:49.079859 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:49.080955 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:49.081274 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:49.081379 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:49.084694 (+ 3315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:56287: BlockingRecv error: recv got EOF from 127.25.31.129:56287 (error 108)
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":74,"threads_started":1}
Aug 06 15:59:50 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:50 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:50 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:50 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:50 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:50 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:50.092401 2872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:50.084818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:53917)
0806 15:59:50.085270 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:50.085287 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:50.085442 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:50.085748 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:50.085751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:50.086008 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:50.086137 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:50.086144 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:50.087281 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:50.087284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:50.088203 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:50.088214 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:50.088855 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:50.088858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:50.088958 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:50.089526 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:50.089541 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:50.092223 (+ 2682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":86,"threads_started":1}
W20260806 15:59:50.092464 2874 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:50.084936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:53917 (local address 127.25.31.190:37967)
0806 15:59:50.085227 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:50.085234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:50.085415 (+ 181us) server_negotiation.cc:407] Connection header received
0806 15:59:50.085562 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:50.085565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:50.085620 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:50.085727 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:50.086284 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:50.087166 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:50.088333 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:50.088722 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:50.088839 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:50.092333 (+ 3494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:53917: BlockingRecv error: recv got EOF from 127.25.31.130:53917 (error 108)
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":168,"thread_start_us":75,"threads_started":1}
I20260806 15:59:50.093073 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:50.084808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:36989)
0806 15:59:50.085270 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:50.085287 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:50.085654 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:50.086025 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:50.086028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:50.086219 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:50.086389 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:50.086398 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:50.087427 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:50.087431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:50.088273 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:50.088287 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:50.088984 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:50.088988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:50.089087 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:50.089700 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:50.089719 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:50.092918 (+ 3199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":82,"threads_started":1}
W20260806 15:59:50.093137 2876 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:50.085097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:36989 (local address 127.25.31.190:37967)
0806 15:59:50.085802 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:50.085806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:50.085817 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:59:50.085849 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:50.085852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:50.085892 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:50.085961 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:50.086604 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:50.087319 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:50.088384 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:50.088813 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:50.088912 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:50.093026 (+ 4114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:36989: BlockingRecv error: recv got EOF from 127.25.31.131:36989 (error 108)
Metrics: {"mutex_wait_us":493,"server-negotiator.queue_time_us":646,"thread_start_us":46,"threads_started":1}
W20260806 15:59:50.093581 2877 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:50.085446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:41913 (local address 127.25.31.190:37967)
0806 15:59:50.086324 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:50.086329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:50.086343 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:59:50.086378 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:50.086381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:50.086425 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:50.086509 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:50.087757 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:50.088243 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:50.089311 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:50.089632 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:50.089749 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:50.093465 (+ 3716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:41913: BlockingRecv error: recv got EOF from 127.25.31.129:41913 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":98,"threads_started":1}
I20260806 15:59:50.093876 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:50.085417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:41913)
0806 15:59:50.086099 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:50.086111 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:50.086177 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:50.087331 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:50.087335 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:50.087541 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:50.087631 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:50.087637 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:50.088397 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:50.088403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:50.089176 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:50.089191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:50.089757 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:50.089764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:50.089863 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:50.090413 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:50.090427 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:50.093763 (+ 3336us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":90,"threads_started":1}
Aug 06 15:59:51 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:51 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:51 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:51.100029 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:51.093184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:33557)
0806 15:59:51.093504 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:51.093537 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:51.093659 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:51.093987 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:51.093990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:51.094231 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:51.094401 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:51.094409 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:51.095303 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:51.095308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:51.096104 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:51.096114 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:51.096775 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:51.096779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:51.096884 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:51.097457 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:51.097472 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:51.099858 (+ 2386us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":90,"threads_started":1}
Aug 06 15:59:51 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:59:51.100057 2879 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:51.093245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:33557 (local address 127.25.31.190:37967)
0806 15:59:51.093491 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:51.093494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:51.093611 (+ 117us) server_negotiation.cc:407] Connection header received
0806 15:59:51.093764 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:51.093770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:51.093849 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:51.093943 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:51.094522 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:51.095186 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:51.096206 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:51.096658 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:51.096765 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:51.099954 (+ 3189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:33557: BlockingRecv error: recv got EOF from 127.25.31.130:33557 (error 108)
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
Aug 06 15:59:51 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:51 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:51.101464 2880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:51.093930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:48407)
0806 15:59:51.094260 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:51.094275 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:51.094411 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:51.095485 (+ 1074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:51.095488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:51.095681 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:51.095786 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:51.095793 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:51.097046 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:51.097052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:51.097863 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:51.097877 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:51.098562 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:51.098566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:51.098670 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:51.099183 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:51.099196 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:51.101315 (+ 2119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":64,"threads_started":1}
W20260806 15:59:51.101610 2881 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:51.093972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:48407 (local address 127.25.31.190:37967)
0806 15:59:51.094198 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:51.094202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:51.094362 (+ 160us) server_negotiation.cc:407] Connection header received
0806 15:59:51.095299 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:51.095326 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:51.095377 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:51.095457 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:51.096231 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:51.096922 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:51.098037 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:51.098428 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:51.098537 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:51.101455 (+ 2918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:48407: BlockingRecv error: recv got EOF from 127.25.31.131:48407 (error 108)
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":141,"thread_start_us":72,"threads_started":1}
I20260806 15:59:51.101775 2883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:51.094615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:34075)
0806 15:59:51.094956 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:51.094971 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:51.095091 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:51.095403 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:51.095409 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:51.095630 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:51.095722 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:51.095729 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:51.096613 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:51.096617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:51.097475 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:51.097487 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:51.097962 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:51.097966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:51.098052 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:51.098556 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:51.098570 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:51.101636 (+ 3066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":80,"threads_started":1}
W20260806 15:59:51.102134 2882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:51.094652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:34075 (local address 127.25.31.190:37967)
0806 15:59:51.094875 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:51.094880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:51.095044 (+ 164us) server_negotiation.cc:407] Connection header received
0806 15:59:51.095181 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:51.095184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:51.095230 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:51.095334 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:51.095857 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:51.096472 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:51.097598 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:51.097841 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:51.097954 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:51.102047 (+ 4093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:34075: BlockingRecv error: recv got EOF from 127.25.31.129:34075 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"spinlock_wait_cycles":2816,"thread_start_us":98,"threads_started":1}
Aug 06 15:59:52 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:52 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:52.107358 2884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:52.100919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:60461)
0806 15:59:52.101212 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:52.101225 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:52.101318 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:52.101560 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:52.101562 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:52.101740 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:52.101894 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:52.101902 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:52.103020 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:52.103024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:52.103626 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:52.103635 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:52.104201 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:52.104204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:52.104283 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:52.104861 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:52.104876 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:52.107152 (+ 2276us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":112,"threads_started":1}
W20260806 15:59:52.107533 2885 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:52.101064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:60461 (local address 127.25.31.190:37967)
0806 15:59:52.101307 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:52.101310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:52.101324 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:59:52.101403 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:52.101405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:52.101448 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:52.101518 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:52.102036 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:52.102887 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:52.103727 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:52.104072 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:52.104172 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:52.107400 (+ 3228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:60461: BlockingRecv error: recv got EOF from 127.25.31.130:60461 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":94,"threads_started":1}
Aug 06 15:59:52 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:52 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:52 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:52 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:52.109112 2886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:52.102262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:54563)
0806 15:59:52.102562 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:52.102578 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:52.102871 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:52.103183 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:52.103186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:52.103371 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:52.103488 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:52.103494 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:52.104716 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:52.104721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:52.105419 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:52.105429 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:52.106089 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:52.106092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:52.106179 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:52.106707 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:52.106723 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:52.108948 (+ 2225us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":88,"threads_started":1}
W20260806 15:59:52.109211 2887 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:52.102328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:54563 (local address 127.25.31.190:37967)
0806 15:59:52.102575 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:52.102580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:52.102672 (+ 92us) server_negotiation.cc:407] Connection header received
0806 15:59:52.102993 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:52.102997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:52.103060 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:52.103156 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:52.103617 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:52.104599 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:52.105548 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:52.105912 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:52.106024 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:52.109091 (+ 3067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:54563: BlockingRecv error: recv got EOF from 127.25.31.131:54563 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":55,"threads_started":1}
I20260806 15:59:52.109582 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:52.102569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:43527)
0806 15:59:52.102831 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:52.102845 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:52.102936 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:52.103998 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:52.104001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:52.104203 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:52.104308 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:52.104316 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:52.105279 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:52.105283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:52.105945 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:52.105955 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:52.106584 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:52.106589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:52.106678 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:52.107147 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:52.107161 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:52.109434 (+ 2273us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":64,"threads_started":1}
W20260806 15:59:52.110078 2889 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:52.102707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:43527 (local address 127.25.31.190:37967)
0806 15:59:52.103077 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:52.103081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:52.103093 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:59:52.103811 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:52.103817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:52.103872 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:52.103967 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:52.104425 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:52.105156 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:52.106097 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:52.106460 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:52.106569 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:52.109953 (+ 3384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:43527: BlockingRecv error: recv got EOF from 127.25.31.129:43527 (error 108)
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":69,"threads_started":1}
Aug 06 15:59:53 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:53 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:53 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:53 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:53 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:53.119092 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:53.110680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:58751)
0806 15:59:53.111518 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:53.111533 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:53.111640 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:53.112610 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:53.112613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:53.112809 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:53.112916 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:53.112924 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:53.113640 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:53.113645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:53.114416 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:53.114427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:53.114882 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:53.114886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:53.114983 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:53.116160 (+ 1177us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:53.116178 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:53.118884 (+ 2706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":87,"threads_started":1}
Aug 06 15:59:53 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:59:53.119184 2895 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:53.110739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:58751 (local address 127.25.31.190:37967)
0806 15:59:53.111034 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:53.111040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:53.111621 (+ 581us) server_negotiation.cc:407] Connection header received
0806 15:59:53.112451 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:53.112452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:53.112486 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:53.112544 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:53.113043 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:53.113519 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:53.114535 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:53.114755 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:53.114874 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:53.119044 (+ 4170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:58751: BlockingRecv error: recv got EOF from 127.25.31.129:58751 (error 108)
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":108,"threads_started":1}
I20260806 15:59:53.119414 2892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:53.110242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:59367)
0806 15:59:53.110566 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:53.110579 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:53.110930 (+ 351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:53.111234 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:53.111237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:53.111451 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:53.111569 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:53.111599 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:53.112451 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:53.112453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:53.113228 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:53.113237 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:53.113692 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:53.113696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:53.113777 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:53.116116 (+ 2339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:53.116132 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:53.119243 (+ 3111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":92,"threads_started":1}
W20260806 15:59:53.119831 2893 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:53.110296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:59367 (local address 127.25.31.190:37967)
0806 15:59:53.111012 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:53.111017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:53.111029 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:59:53.111067 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:53.111070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:53.111117 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:53.111189 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:53.111682 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:53.112347 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:53.113361 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:53.113580 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:53.113686 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:53.119704 (+ 6018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:59367: BlockingRecv error: recv got EOF from 127.25.31.131:59367 (error 108)
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":61,"threads_started":1}
I20260806 15:59:53.119858 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:53.108147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:33883)
0806 15:59:53.108422 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:53.108436 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:53.108526 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:53.108771 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:53.108773 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:53.108967 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:53.109072 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:53.109078 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:53.109923 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:53.109927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:53.110812 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:53.110827 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:53.111382 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:53.111385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:53.111450 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:53.116261 (+ 4811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:53.116277 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:53.119684 (+ 3407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":97,"threads_started":1}
W20260806 15:59:53.119877 2891 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:53.108258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:33883 (local address 127.25.31.190:37967)
0806 15:59:53.108507 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:53.108511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:53.108524 (+ 13us) server_negotiation.cc:407] Connection header received
0806 15:59:53.108612 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:53.108614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:53.108656 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:53.108725 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:53.109198 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:53.109779 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:53.110933 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:53.111243 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:53.111355 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:53.119781 (+ 8426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:33883: BlockingRecv error: recv got EOF from 127.25.31.130:33883 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":91,"threads_started":1}
Aug 06 15:59:54 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:54 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:54 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:54.126631 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:54.120253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:60507)
0806 15:59:54.120553 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:54.120569 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:54.120674 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:54.120945 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:54.120948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:54.121191 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:54.121305 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:54.121311 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:54.122367 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:54.122372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:54.122942 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:54.122950 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:54.123482 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:54.123485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:54.123573 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:54.124177 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:54.124192 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:54.126461 (+ 2269us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":96,"threads_started":1}
Aug 06 15:59:54 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:59:54.126796 2896 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:54.120286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:60507 (local address 127.25.31.190:37967)
0806 15:59:54.120599 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:54.120605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:54.120621 (+ 16us) server_negotiation.cc:407] Connection header received
0806 15:59:54.120781 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:54.120784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:54.120836 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:54.120927 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:54.121433 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:54.122242 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:54.123059 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:54.123352 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:54.123450 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:54.126616 (+ 3166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:60507: BlockingRecv error: recv got EOF from 127.25.31.131:60507 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":234,"thread_start_us":69,"threads_started":1}
Aug 06 15:59:54 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:54 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:54.127640 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:54.120713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:48153)
0806 15:59:54.121042 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:54.121056 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:54.121149 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:54.121789 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:54.121792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:54.122031 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:54.122126 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:54.122132 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:54.123355 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:54.123359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:54.124155 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:54.124166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:54.124822 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:54.124828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:54.124936 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:54.125499 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:54.125511 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:54.127473 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":64,"threads_started":1}
W20260806 15:59:54.127785 2901 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:54.121326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:48153 (local address 127.25.31.190:37967)
0806 15:59:54.121534 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:54.121542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:54.121555 (+ 13us) server_negotiation.cc:407] Connection header received
0806 15:59:54.121590 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:54.121593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:54.121632 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:54.122218 (+ 586us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:54.122339 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:54.123236 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:54.124277 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:54.124692 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:54.125169 (+ 477us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:54.127637 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:48153: BlockingRecv error: recv got EOF from 127.25.31.130:48153 (error 108)
Metrics: {"mutex_wait_us":412,"server-negotiator.queue_time_us":147,"thread_start_us":76,"threads_started":1}
I20260806 15:59:54.128159 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:54.120414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:48883)
0806 15:59:54.121465 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:54.121480 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:54.121565 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:54.121899 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:54.121902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:54.122088 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:54.122189 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:54.122195 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:54.123395 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:54.123420 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:54.124205 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:54.124217 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:54.124843 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:54.124847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:54.124944 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:54.125607 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:54.125626 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:54.128062 (+ 2436us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":68,"threads_started":1}
W20260806 15:59:54.128283 2900 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:54.120810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:48883 (local address 127.25.31.190:37967)
0806 15:59:54.121159 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:54.121164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:54.121696 (+ 532us) server_negotiation.cc:407] Connection header received
0806 15:59:54.121731 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:54.121736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:54.121779 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:54.121855 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:54.122339 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:54.123243 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:54.124322 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:54.124692 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:54.124820 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:54.128172 (+ 3352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:48883: BlockingRecv error: recv got EOF from 127.25.31.129:48883 (error 108)
Metrics: {"mutex_wait_us":111,"server-negotiator.queue_time_us":283,"thread_start_us":51,"threads_started":1}
Aug 06 15:59:55 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:55 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:55 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:55 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:55 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:55 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:55.135309 2904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:55.128381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:53675)
0806 15:59:55.128655 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:55.128669 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:55.128761 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:55.129037 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:55.129043 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:55.129258 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:55.129398 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:55.129406 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:55.130265 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:55.130268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:55.130998 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:55.131008 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:55.131775 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:55.131778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:55.131875 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:55.132448 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:55.132464 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:55.135131 (+ 2667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":69,"threads_started":1}
W20260806 15:59:55.135422 2905 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:55.128397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:53675 (local address 127.25.31.190:37967)
0806 15:59:55.128688 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:55.128694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:55.128713 (+ 19us) server_negotiation.cc:407] Connection header received
0806 15:59:55.128859 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:55.128862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:55.128907 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:55.128992 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:55.129516 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:55.130176 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:55.131127 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:55.131588 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:55.131706 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:55.135272 (+ 3566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:53675: BlockingRecv error: recv got EOF from 127.25.31.130:53675 (error 108)
Metrics: {"mutex_wait_us":150,"server-negotiator.queue_time_us":223,"thread_start_us":75,"threads_started":1}
I20260806 15:59:55.135690 2902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:55.127601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:41335)
0806 15:59:55.128073 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:55.128087 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:55.128221 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:55.128752 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:55.128755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:55.129219 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:55.129352 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:55.129363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:55.130191 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:55.130197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:55.131106 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:55.131116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:55.132453 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:55.132456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:55.132518 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:55.132989 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:55.133004 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:55.135562 (+ 2558us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":117,"threads_started":1}
W20260806 15:59:55.135716 2903 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:55.127688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:41335 (local address 127.25.31.190:37967)
0806 15:59:55.127942 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:55.127947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:55.128186 (+ 239us) server_negotiation.cc:407] Connection header received
0806 15:59:55.128390 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:55.128394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:55.128454 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:55.128549 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:55.129445 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:55.130097 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:55.131198 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:55.131587 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:55.131694 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:55.135579 (+ 3885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:41335: BlockingRecv error: recv got EOF from 127.25.31.131:41335 (error 108)
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":182,"thread_start_us":79,"threads_started":1}
I20260806 15:59:55.136060 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:55.128962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:55593)
0806 15:59:55.129217 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:55.129231 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:55.129315 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:55.130154 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:55.130160 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:55.130421 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:55.130558 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:55.130567 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:55.131480 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:55.131484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:55.132269 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:55.132284 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:55.132886 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:55.132892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:55.132998 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:55.133490 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:55.133511 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:55.135924 (+ 2413us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":78,"threads_started":1}
W20260806 15:59:55.136428 2907 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:55.129026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:55593 (local address 127.25.31.190:37967)
0806 15:59:55.129287 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:55.129291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:55.129306 (+ 15us) server_negotiation.cc:407] Connection header received
0806 15:59:55.129405 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:55.129409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:55.129992 (+ 583us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:55.130087 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:55.130684 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:55.131370 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:55.132422 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:55.132752 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:55.132859 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:55.136349 (+ 3490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:55593: BlockingRecv error: recv got EOF from 127.25.31.129:55593 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":58,"threads_started":1}
Aug 06 15:59:56 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:56 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:56 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:56 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:56.143678 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:56.136563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:57937)
0806 15:59:56.136876 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:56.136905 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:56.137014 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:56.137477 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:56.137480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:56.137722 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:56.137854 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:56.137862 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:56.139151 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:56.139155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:56.139923 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:56.139932 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:56.140397 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:56.140401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:56.140487 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:56.141093 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:56.141109 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:56.143507 (+ 2398us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":60,"threads_started":1}
Aug 06 15:59:56 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:59:56.143784 2911 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:56.136756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:57937 (local address 127.25.31.190:37967)
0806 15:59:56.137248 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:56.137252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:56.137266 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:59:56.137305 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:56.137309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:56.137360 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:56.137486 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:56.138136 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:56.139041 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:56.140037 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:56.140277 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:56.140387 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:56.143653 (+ 3266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:57937: BlockingRecv error: recv got EOF from 127.25.31.131:57937 (error 108)
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":396,"thread_start_us":70,"threads_started":1}
I20260806 15:59:56.144060 2909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:56.136503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:51879)
0806 15:59:56.137028 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:56.137047 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:56.137156 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:56.138165 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:56.138171 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:56.138492 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:56.138604 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:56.138612 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:56.139504 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:56.139508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:56.140393 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:56.140403 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:56.141063 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:56.141070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:56.141164 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:56.141656 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:56.141677 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:56.143856 (+ 2179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":88,"threads_started":1}
Aug 06 15:59:56 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:59:56.144508 2912 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:56.136818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:51879 (local address 127.25.31.190:37967)
0806 15:59:56.137362 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:56.137366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:56.137378 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:59:56.137413 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:56.137975 (+ 562us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:56.138048 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:56.138133 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:56.138734 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:56.139397 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:56.140527 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:56.140946 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:56.141058 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:56.144288 (+ 3230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:51879: BlockingRecv error: recv got EOF from 127.25.31.130:51879 (error 108)
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":431,"spinlock_wait_cycles":2688,"thread_start_us":83,"threads_started":1}
I20260806 15:59:56.144976 2913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:56.136825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:47769)
0806 15:59:56.137462 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:56.137476 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:56.137556 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:56.137835 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:56.137838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:56.138212 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:56.138575 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:56.138583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:56.139694 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:56.139697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:56.140551 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:56.140561 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:56.141648 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:56.141652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:56.141765 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:56.142254 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:56.142266 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:56.144879 (+ 2613us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":79,"threads_started":1}
W20260806 15:59:56.145207 2914 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:56.137201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:47769 (local address 127.25.31.190:37967)
0806 15:59:56.137554 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:56.137558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:56.137570 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:59:56.137658 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:56.137662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:56.137708 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:56.137955 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:56.138705 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:56.139580 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:56.140668 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:56.140996 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:56.141096 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:56.145073 (+ 3977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:47769: BlockingRecv error: recv got EOF from 127.25.31.129:47769 (error 108)
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":286,"spinlock_wait_cycles":19712,"thread_start_us":80,"threads_started":1}
Aug 06 15:59:57 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:57 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:57 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:57 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:57.151305 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:57.145002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:41959)
0806 15:59:57.145319 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:57.145333 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:57.145424 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:57.145836 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:57.145839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:57.146112 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:57.146237 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:57.146244 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:57.147092 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:57.147097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:57.147800 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:57.147810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:57.148316 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:57.148319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:57.148407 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:57.148983 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:57.148998 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:57.151125 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":76,"threads_started":1}
W20260806 15:59:57.151451 2918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:57.145144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:41959 (local address 127.25.31.190:37967)
0806 15:59:57.145598 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:57.145603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:57.145617 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:59:57.145653 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:57.145656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:57.145698 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:57.145788 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:57.146375 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:57.146983 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:57.147929 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:57.148200 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:57.148311 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:57.151293 (+ 2982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:41959: BlockingRecv error: recv got EOF from 127.25.31.130:41959 (error 108)
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":55,"threads_started":1}
Aug 06 15:59:57 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:57.152148 2915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:57.144723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:44995)
0806 15:59:57.145052 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:57.145073 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:57.145195 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:57.145934 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:57.145937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:57.146154 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:57.146288 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:57.146295 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:57.147163 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:57.147167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:57.147935 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:57.147946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:57.148523 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:57.148526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:57.148633 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:57.149125 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:57.149147 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:57.152004 (+ 2857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":86,"threads_started":1}
Aug 06 15:59:57 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:59:57.152271 2916 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:57.144825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:44995 (local address 127.25.31.190:37967)
0806 15:59:57.145077 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:57.145083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:57.145135 (+ 52us) server_negotiation.cc:407] Connection header received
0806 15:59:57.145324 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:57.145329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:57.145402 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:57.145486 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:57.146429 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:57.147044 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:57.148053 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:57.148400 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:57.148507 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:57.152130 (+ 3623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:44995: BlockingRecv error: recv got EOF from 127.25.31.131:44995 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":168,"thread_start_us":70,"threads_started":1}
I20260806 15:59:57.152915 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:57.145687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:58713)
0806 15:59:57.146024 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:57.146039 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:57.146139 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:57.147140 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:57.147146 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:57.147340 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:57.147449 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:57.147455 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:57.148433 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:57.148437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:57.149256 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:57.149268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:57.149886 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:57.149890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:57.149980 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:57.150493 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:57.150515 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:57.152726 (+ 2211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":70,"threads_started":1}
W20260806 15:59:57.152982 2920 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:57.145718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:58713 (local address 127.25.31.190:37967)
0806 15:59:57.145939 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:57.145943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:57.146198 (+ 255us) server_negotiation.cc:407] Connection header received
0806 15:59:57.146239 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:57.146242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:57.146288 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:57.146398 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:57.147576 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:57.148295 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:57.149395 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:57.149761 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:57.149885 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:57.152846 (+ 2961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:58713: BlockingRecv error: recv got EOF from 127.25.31.129:58713 (error 108)
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":145,"thread_start_us":69,"threads_started":1}
Aug 06 15:59:58 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:58 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:58 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.131@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:58 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.130@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
W20260806 15:59:58.161002 2922 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:58.152281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:48507 (local address 127.25.31.190:37967)
0806 15:59:58.152565 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:58.152570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:58.152587 (+ 17us) server_negotiation.cc:407] Connection header received
0806 15:59:58.152694 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:58.152697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:58.152760 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:58.152855 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:58.153485 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:58.154639 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:58.156359 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:58.156728 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:58.156817 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:58.160866 (+ 4049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.130:48507: BlockingRecv error: recv got EOF from 127.25.31.130:48507 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":85,"threads_started":1}
W20260806 15:59:58.161065 2924 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:58.153213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:35179 (local address 127.25.31.190:37967)
0806 15:59:58.153756 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:58.153761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:58.154175 (+ 414us) server_negotiation.cc:407] Connection header received
0806 15:59:58.154324 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:58.154328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:58.154375 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:58.154491 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:58.155204 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:58.155985 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:58.157131 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:58.157464 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:58.157567 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:58.160907 (+ 3340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.131:35179: BlockingRecv error: recv got EOF from 127.25.31.131:35179 (error 108)
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":387,"threads_started":1}
I20260806 15:59:58.161095 2923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:58.152870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.131:35179)
0806 15:59:58.154103 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:58.154118 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:58.154220 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:58.154531 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:58.154534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:58.154740 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:58.155084 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:58.155099 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:58.156121 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:58.156124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:58.157023 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:58.157034 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:58.157588 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:58.157591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:58.157683 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:58.158186 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:58.158204 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:58.160749 (+ 2545us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1158,"thread_start_us":83,"threads_started":1}
I20260806 15:59:58.161767 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:58.152191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.130:48507)
0806 15:59:58.152460 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:58.152475 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:58.152579 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:58.152893 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:58.152898 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:58.153182 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:58.153341 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:58.153353 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:58.155284 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:58.155288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:58.156255 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:58.156267 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:58.156849 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:58.156855 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:58.156974 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:58.157941 (+ 967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:58.157957 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:58.161544 (+ 3587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":86,"threads_started":1}
Aug 06 15:59:58 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
Aug 06 15:59:58 dist-test-slave-tm1g krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.31.129@KRBTEST.COM for oryx/127.25.31.190@KRBTEST.COM, Server not found in Kerberos database
I20260806 15:59:58.163110 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:58.153832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37967 (local address 127.25.31.129:37359)
0806 15:59:58.155082 (+ 1250us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:58.155096 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:58.155202 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:58.156271 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:58.156277 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:58.157073 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:58.157197 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:58.157208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:58.158410 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:58.158417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:58.159445 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:58.159460 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:58.160260 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:58.160266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:58.160379 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:58.160933 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:59:58.160947 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:58.162959 (+ 2012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.31.190:37967: Server oryx/127.25.31.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1173,"thread_start_us":70,"threads_started":1}
W20260806 15:59:58.163479 2925 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:58.153828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:37359 (local address 127.25.31.190:37967)
0806 15:59:58.154052 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:58.154056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:58.156083 (+ 2027us) server_negotiation.cc:407] Connection header received
0806 15:59:58.156111 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:58.156114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:58.156154 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:58.156228 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:58.157311 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:58.158115 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:58.159600 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:58.160009 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:58.160120 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:58.163123 (+ 3003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.31.129:37359: BlockingRecv error: recv got EOF from 127.25.31.129:37359 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":81,"threads_started":1}
I20260806 15:59:58.591570 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 2418
I20260806 15:59:58.598640 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 2573
I20260806 15:59:58.604259 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 2708
I20260806 15:59:58.610529 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 2346
2026-08-06T15:59:58Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18002 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T15:59:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:58Z Disabled control of system clock
I20260806 15:59:58.641578 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:38999
--webserver_interface=127.25.31.190
--webserver_port=0
--builtin_ntp_servers=127.25.31.148:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:38999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 15:59:58.750237 2930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:58.750504 2930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:58.750540 2930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:58.754369 2930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:58.754421 2930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:58.754439 2930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:58.754456 2930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:58.754472 2930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:58.759078 2930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:38999
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:38999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:58.760004 2930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:58.761296 2930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:58.769991 2935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:58.770030 2930 server_base.cc:1034] running on GCE node
W20260806 15:59:58.770028 2936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:58.769997 2938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:58.770819 2930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:58.771836 2930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:58.773003 2930 hybrid_clock.cc:648] HybridClock initialized: now 1786031998772988 us; error 26 us; skew 500 ppm
I20260806 15:59:58.774747 2930 webserver.cc:460] Webserver started at http://127.25.31.190:34361/ using document root <none> and password file <none>
I20260806 15:59:58.775247 2930 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:58.775303 2930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:58.775471 2930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:58.777359 2930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "ebdf40f2f01a43daaa03cbf183aa8657"
format_stamp: "Formatted at 2026-08-06 15:59:58 on dist-test-slave-tm1g"
server_key: "9db985b32c6d39c9e9e95f962898d95a"
server_key_iv: "720ce13658df2921e26488d443b423a1"
server_key_version: "encryptionkey@0"
I20260806 15:59:58.777868 2930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "ebdf40f2f01a43daaa03cbf183aa8657"
format_stamp: "Formatted at 2026-08-06 15:59:58 on dist-test-slave-tm1g"
server_key: "9db985b32c6d39c9e9e95f962898d95a"
server_key_iv: "720ce13658df2921e26488d443b423a1"
server_key_version: "encryptionkey@0"
I20260806 15:59:58.781288 2930 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 15:59:58.783600 2944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:58.784754 2930 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 15:59:58.784898 2930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "ebdf40f2f01a43daaa03cbf183aa8657"
format_stamp: "Formatted at 2026-08-06 15:59:58 on dist-test-slave-tm1g"
server_key: "9db985b32c6d39c9e9e95f962898d95a"
server_key_iv: "720ce13658df2921e26488d443b423a1"
server_key_version: "encryptionkey@0"
I20260806 15:59:58.785009 2930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:58.803406 2930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:58.804055 2930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:58.804243 2930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:58.810828 2930 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:38999
I20260806 15:59:58.810860 2998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:38999 every 8 connection(s)
I20260806 15:59:58.811721 2930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:58.814893 2999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:58.816826 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 2930
I20260806 15:59:58.816957 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:58.817281 25726 external_mini_cluster.cc:1428] Setting key b793af99064713e3c3c375bc02b2f370
I20260806 15:59:58.821038 2999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657: Bootstrap starting.
I20260806 15:59:58.823257 2999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:58.823992 2999 log.cc:824] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:58.826062 2999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657: No bootstrap required, opened a new log
I20260806 15:59:58.826812 3002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:58.818738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44822 (local address 127.25.31.190:38999)
0806 15:59:58.819371 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:58.819386 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:58.819432 (+ 46us) server_negotiation.cc:407] Connection header received
0806 15:59:58.820283 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:58.820318 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:58.820697 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:58.821030 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:58.821469 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:58.822503 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:58.823411 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:58.823881 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:58.824050 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:58.825039 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:58.825044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:58.825057 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:58.825446 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:58.825508 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:58.825527 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:58.825639 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:58.825757 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 15:59:58.826010 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":123,"threads_started":1}
I20260806 15:59:58.829978 2999 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebdf40f2f01a43daaa03cbf183aa8657" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 38999 } }
I20260806 15:59:58.830219 2999 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:58.830336 2999 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebdf40f2f01a43daaa03cbf183aa8657, State: Initialized, Role: FOLLOWER
I20260806 15:59:58.830844 2999 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebdf40f2f01a43daaa03cbf183aa8657" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 38999 } }
I20260806 15:59:58.831027 2999 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:58.831095 2999 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:58.831189 2999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:58.832075 2999 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebdf40f2f01a43daaa03cbf183aa8657" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 38999 } }
I20260806 15:59:58.832413 2999 leader_election.cc:302] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebdf40f2f01a43daaa03cbf183aa8657; no voters:
I20260806 15:59:58.832720 2999 leader_election.cc:288] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:58.832860 3005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:58.833086 3005 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [term 1 LEADER]: Becoming Leader. State: Replica: ebdf40f2f01a43daaa03cbf183aa8657, State: Running, Role: LEADER
I20260806 15:59:58.833421 3005 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebdf40f2f01a43daaa03cbf183aa8657" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 38999 } }
I20260806 15:59:58.833899 2999 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:58.835111 3006 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebdf40f2f01a43daaa03cbf183aa8657" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebdf40f2f01a43daaa03cbf183aa8657" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 38999 } } }
I20260806 15:59:58.835111 3007 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebdf40f2f01a43daaa03cbf183aa8657. Latest consensus state: current_term: 1 leader_uuid: "ebdf40f2f01a43daaa03cbf183aa8657" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebdf40f2f01a43daaa03cbf183aa8657" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 38999 } } }
I20260806 15:59:58.835285 3006 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:58.835321 3007 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:58.835675 3014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:58.838369 3014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:58.843683 3014 catalog_manager.cc:1347] Generated new cluster ID: 4df994cca9c3441fac394e30ea4b1b2c
I20260806 15:59:58.843740 3014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:58.859169 3014 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:58.860298 3014 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:58.873853 3014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebdf40f2f01a43daaa03cbf183aa8657: Generated new TSK 0
I20260806 15:59:58.874650 3014 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:59:58.880556 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:38999
--builtin_ntp_servers=127.25.31.148:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 15:59:58.991834 3024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:58.992071 3024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:58.992107 3024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:58.996134 3024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:58.996181 3024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:58.996230 3024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:59.000820 3024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:38999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:59.001844 3024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:59.003224 3024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:59.012739 3032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:59.012741 3029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:59.012799 3030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:59.013000 3024 server_base.cc:1034] running on GCE node
I20260806 15:59:59.013494 3024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:59.014107 3024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:59.015295 3024 hybrid_clock.cc:648] HybridClock initialized: now 1786031999015284 us; error 52 us; skew 500 ppm
I20260806 15:59:59.017096 3024 webserver.cc:460] Webserver started at http://127.25.31.129:42641/ using document root <none> and password file <none>
I20260806 15:59:59.017683 3024 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:59.017776 3024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:59.018013 3024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:59.020090 3024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "793a78e85db94899a9c292b1ae9bf3ac"
format_stamp: "Formatted at 2026-08-06 15:59:59 on dist-test-slave-tm1g"
server_key: "dd71de7dced823db57f85292331484ef"
server_key_iv: "9e04995ae43920c5d3745faf8d5175f5"
server_key_version: "encryptionkey@0"
I20260806 15:59:59.020659 3024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "793a78e85db94899a9c292b1ae9bf3ac"
format_stamp: "Formatted at 2026-08-06 15:59:59 on dist-test-slave-tm1g"
server_key: "dd71de7dced823db57f85292331484ef"
server_key_iv: "9e04995ae43920c5d3745faf8d5175f5"
server_key_version: "encryptionkey@0"
I20260806 15:59:59.024230 3024 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 15:59:59.026824 3038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:59.027972 3024 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:59.028123 3024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "793a78e85db94899a9c292b1ae9bf3ac"
format_stamp: "Formatted at 2026-08-06 15:59:59 on dist-test-slave-tm1g"
server_key: "dd71de7dced823db57f85292331484ef"
server_key_iv: "9e04995ae43920c5d3745faf8d5175f5"
server_key_version: "encryptionkey@0"
I20260806 15:59:59.028237 3024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:59.042784 3024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:59.043402 3024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:59.043601 3024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:59.044162 3024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:59.045068 3024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:59.045137 3024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:59.045208 3024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:59.045251 3024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:59.054183 3024 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:39279
I20260806 15:59:59.054207 3154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:39279 every 8 connection(s)
I20260806 15:59:59.055087 3024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:59.057000 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 3024
I20260806 15:59:59.057113 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 15:59:59.057441 25726 external_mini_cluster.cc:1428] Setting key f75bf457e4f209f17dd278b8193eaec5
I20260806 15:59:59.059499 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:38999
--builtin_ntp_servers=127.25.31.148:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 15:59:59.064864 3002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:59.056849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:56791 (local address 127.25.31.190:38999)
0806 15:59:59.056995 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:59.056999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:59.058774 (+ 1775us) server_negotiation.cc:407] Connection header received
0806 15:59:59.059860 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:59.059863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:59.059918 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:59.060002 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:59.061006 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:59.061536 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:59.062917 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:59.063159 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:59.063274 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:59.064365 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:59.064369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:59.064372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:59.064384 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:59.064413 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:59.064421 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:59.064500 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:59.064665 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 15:59:59.064727 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 15:59:59.065692 3157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:59.057340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:38999 (local address 127.25.31.129:56791)
0806 15:59:59.058110 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:59.058151 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:59.059633 (+ 1482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:59.060176 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:59.060183 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:59.060264 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:59.060835 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:59.060858 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:59.061691 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:59.061697 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:59.062786 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:59.062797 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:59.063318 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:59.063322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:59.063487 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:59.064111 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:59:59.064113 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:59.064151 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:59:59.064212 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:59.064504 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:59.064516 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:59.064524 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 15:59:59.064664 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 15:59:59.064936 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":150,"threads_started":1}
I20260806 15:59:59.066951 3155 heartbeater.cc:347] Connected to a master server at 127.25.31.190:38999
I20260806 15:59:59.067194 3155 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:59.067684 3155 heartbeater.cc:511] Master 127.25.31.190:38999 requested a full tablet report, sending...
I20260806 15:59:59.069378 2961 ts_manager.cc:194] Registered new tserver with Master: 793a78e85db94899a9c292b1ae9bf3ac (127.25.31.129:39279)
I20260806 15:59:59.071054 2961 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.31.129:56791
W20260806 15:59:59.171082 3158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:59.171309 3158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:59.171344 3158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:59.175330 3158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:59.175374 3158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:59.175422 3158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:59.179891 3158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:38999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:59.180791 3158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:59.182051 3158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:59.191634 3165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:59.191771 3158 server_base.cc:1034] running on GCE node
W20260806 15:59:59.191721 3164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:59.191635 3167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:59.192298 3158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:59.192842 3158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:59.194051 3158 hybrid_clock.cc:648] HybridClock initialized: now 1786031999194025 us; error 29 us; skew 500 ppm
I20260806 15:59:59.195856 3158 webserver.cc:460] Webserver started at http://127.25.31.130:46537/ using document root <none> and password file <none>
I20260806 15:59:59.196416 3158 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:59.196511 3158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:59.196744 3158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:59.198791 3158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "ecdeff1bf7604f1eb1052e9ab7c7175a"
format_stamp: "Formatted at 2026-08-06 15:59:59 on dist-test-slave-tm1g"
server_key: "5f7201ecd74e5890fee524158d1a5c7d"
server_key_iv: "33be097d2de3c9238ec1838401885476"
server_key_version: "encryptionkey@0"
I20260806 15:59:59.199355 3158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "ecdeff1bf7604f1eb1052e9ab7c7175a"
format_stamp: "Formatted at 2026-08-06 15:59:59 on dist-test-slave-tm1g"
server_key: "5f7201ecd74e5890fee524158d1a5c7d"
server_key_iv: "33be097d2de3c9238ec1838401885476"
server_key_version: "encryptionkey@0"
I20260806 15:59:59.202955 3158 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 15:59:59.205451 3173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:59.206689 3158 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 15:59:59.206822 3158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "ecdeff1bf7604f1eb1052e9ab7c7175a"
format_stamp: "Formatted at 2026-08-06 15:59:59 on dist-test-slave-tm1g"
server_key: "5f7201ecd74e5890fee524158d1a5c7d"
server_key_iv: "33be097d2de3c9238ec1838401885476"
server_key_version: "encryptionkey@0"
I20260806 15:59:59.206934 3158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:59.218941 3158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:59.219607 3158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:59.219810 3158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:59.220382 3158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:59.221360 3158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:59.221433 3158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:59.221506 3158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:59.221560 3158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:59.230232 3158 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:42159
I20260806 15:59:59.230261 3288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:42159 every 8 connection(s)
I20260806 15:59:59.231132 3158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 15:59:59.235957 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 3158
I20260806 15:59:59.236074 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 15:59:59.236377 25726 external_mini_cluster.cc:1428] Setting key 75582bc6fd6472bad4cf0e3fa7307657
I20260806 15:59:59.238340 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:38999
--builtin_ntp_servers=127.25.31.148:34985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 15:59:59.240986 3002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:59.232963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:50765 (local address 127.25.31.190:38999)
0806 15:59:59.233101 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:59.233105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:59.234197 (+ 1092us) server_negotiation.cc:407] Connection header received
0806 15:59:59.235764 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:59.235768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:59.235829 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:59.235941 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:59.236945 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:59.237668 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:59.238985 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:59.239219 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:59.239332 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:59.240444 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:59.240448 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:59.240451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:59.240463 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:59.240484 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:59.240493 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:59.240577 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:59.240772 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 15:59:59.240833 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 15:59:59.241852 3291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:59.233317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:38999 (local address 127.25.31.130:50765)
0806 15:59:59.234049 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:59.234088 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:59.235555 (+ 1467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:59.236158 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:59.236167 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:59.236264 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:59.236788 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:59.236801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:59.237797 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:59.237802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:59.238855 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:59.238865 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:59.239370 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:59.239374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:59.239543 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:59.240189 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:59:59.240190 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:59.240229 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:59:59.240289 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:59.240591 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:59.240603 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:59.240615 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 15:59:59.240764 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 15:59:59.241035 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":149,"threads_started":1}
I20260806 15:59:59.243165 3289 heartbeater.cc:347] Connected to a master server at 127.25.31.190:38999
I20260806 15:59:59.243391 3289 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:59.243883 3289 heartbeater.cc:511] Master 127.25.31.190:38999 requested a full tablet report, sending...
I20260806 15:59:59.244927 2961 ts_manager.cc:194] Registered new tserver with Master: ecdeff1bf7604f1eb1052e9ab7c7175a (127.25.31.130:42159)
I20260806 15:59:59.245502 2961 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.31.130:50765
W20260806 15:59:59.349484 3292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:59.349758 3292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:59.349828 3292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:59.353915 3292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:59.353984 3292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:59.354071 3292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 15:59:59.358736 3292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:34985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:38999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:59.359743 3292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:59.361083 3292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:59.370520 3299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:59.370489 3298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:59.370514 3301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:59.370550 3292 server_base.cc:1034] running on GCE node
I20260806 15:59:59.371158 3292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:59.371755 3292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:59.372953 3292 hybrid_clock.cc:648] HybridClock initialized: now 1786031999372935 us; error 30 us; skew 500 ppm
I20260806 15:59:59.374913 3292 webserver.cc:460] Webserver started at http://127.25.31.131:34845/ using document root <none> and password file <none>
I20260806 15:59:59.375499 3292 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:59.375591 3292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:59.375823 3292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:59.377763 3292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "7c4fe9350a5441bea15dd7cd97aa1a47"
format_stamp: "Formatted at 2026-08-06 15:59:59 on dist-test-slave-tm1g"
server_key: "6f9739ab2deb612bf577ebd9f4aec08b"
server_key_iv: "1654942899df7dfd3add1f53734d6e19"
server_key_version: "encryptionkey@0"
I20260806 15:59:59.378373 3292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "7c4fe9350a5441bea15dd7cd97aa1a47"
format_stamp: "Formatted at 2026-08-06 15:59:59 on dist-test-slave-tm1g"
server_key: "6f9739ab2deb612bf577ebd9f4aec08b"
server_key_iv: "1654942899df7dfd3add1f53734d6e19"
server_key_version: "encryptionkey@0"
I20260806 15:59:59.381800 3292 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 15:59:59.384212 3307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:59.385339 3292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:59:59.385469 3292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "7c4fe9350a5441bea15dd7cd97aa1a47"
format_stamp: "Formatted at 2026-08-06 15:59:59 on dist-test-slave-tm1g"
server_key: "6f9739ab2deb612bf577ebd9f4aec08b"
server_key_iv: "1654942899df7dfd3add1f53734d6e19"
server_key_version: "encryptionkey@0"
I20260806 15:59:59.385576 3292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:59.395560 3292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:59.396184 3292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:59.396368 3292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:59.396903 3292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:59.397778 3292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:59.397851 3292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:59.397938 3292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:59.397987 3292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:59.406378 3292 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:46705
I20260806 15:59:59.406405 3432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:46705 every 8 connection(s)
I20260806 15:59:59.407243 3292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 15:59:59.414543 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 3292
I20260806 15:59:59.414664 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 15:59:59.414961 25726 external_mini_cluster.cc:1428] Setting key 45bd138107c14b01df5dc1f3de84eaa1
I20260806 15:59:59.416344 3002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:59.409058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:47751 (local address 127.25.31.190:38999)
0806 15:59:59.409191 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:59.409198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:59.410208 (+ 1010us) server_negotiation.cc:407] Connection header received
0806 15:59:59.411665 (+ 1457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:59.411668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:59.411721 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:59.411795 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:59.412631 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:59.413152 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:59.414275 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:59:59.414566 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:59:59.414681 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:59.415858 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:59.415862 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:59.415864 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:59.415876 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:59.415895 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:59.415903 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:59.415984 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:59.416140 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 15:59:59.416203 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 15:59:59.417168 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:59.409363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:38999 (local address 127.25.31.131:47751)
0806 15:59:59.410058 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:59.410097 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:59.411478 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:59.411951 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:59.411957 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:59.412035 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:59.412509 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:59.412521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:59.413269 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:59.413272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:59.414170 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:59:59.414179 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:59:59.414729 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:59:59.414733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:59:59.414910 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:59:59.415594 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:59:59.415595 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:59.415634 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:59:59.415695 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:59.415986 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:59.415998 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:59.416006 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 15:59:59.416139 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 15:59:59.416412 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":114,"threads_started":1}
I20260806 15:59:59.418208 3433 heartbeater.cc:347] Connected to a master server at 127.25.31.190:38999
I20260806 15:59:59.418462 3433 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:59.418967 3433 heartbeater.cc:511] Master 127.25.31.190:38999 requested a full tablet report, sending...
I20260806 15:59:59.419939 2961 ts_manager.cc:194] Registered new tserver with Master: 7c4fe9350a5441bea15dd7cd97aa1a47 (127.25.31.131:46705)
I20260806 15:59:59.420516 2961 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.31.131:47751
I20260806 15:59:59.428812 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 15:59:59.431761 3002 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:59:59.430831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44836 (local address 127.25.31.190:38999)
0806 15:59:59.430950 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:59.430954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:59.431126 (+ 172us) server_negotiation.cc:407] Connection header received
0806 15:59:59.431212 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:59.431214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:59.431252 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:59.431365 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:59.431591 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44836: BlockingRecv error: recv got EOF from 127.0.0.1:44836 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 15:59:59.432088 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 3024
I20260806 15:59:59.438705 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 3158
I20260806 15:59:59.444923 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 3292
I20260806 15:59:59.450807 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 2930
2026-08-06T15:59:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (838 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T15:59:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:59:59Z Disabled control of system clock
I20260806 15:59:59.480350 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:46007
--webserver_interface=127.25.31.190
--webserver_port=0
--builtin_ntp_servers=127.25.31.148:44089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:46007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 15:59:59.589442 3444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:59.589679 3444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:59.589715 3444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:59.593526 3444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:59:59.593573 3444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:59.593590 3444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:59.593605 3444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:59:59.593621 3444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:59:59.598109 3444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:46007
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:46007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:59.599193 3444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:59.600461 3444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:59:59.608680 3452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:59.608678 3449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:59.608686 3450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:59.608863 3444 server_base.cc:1034] running on GCE node
I20260806 15:59:59.609465 3444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:59.610525 3444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:59.611711 3444 hybrid_clock.cc:648] HybridClock initialized: now 1786031999611697 us; error 28 us; skew 500 ppm
I20260806 15:59:59.613518 3444 webserver.cc:460] Webserver started at http://127.25.31.190:32781/ using document root <none> and password file <none>
I20260806 15:59:59.614086 3444 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:59.614177 3444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:59.614454 3444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:59.616428 3444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "de2bd7c7aa524dfd8da27ee5da854e65"
format_stamp: "Formatted at 2026-08-06 15:59:59 on dist-test-slave-tm1g"
server_key: "a26cd3a603139b2ec2bdb471060d78e5"
server_key_iv: "8d11e10c78ca09c4dbc7dff849ad08c9"
server_key_version: "encryptionkey@0"
I20260806 15:59:59.616995 3444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "de2bd7c7aa524dfd8da27ee5da854e65"
format_stamp: "Formatted at 2026-08-06 15:59:59 on dist-test-slave-tm1g"
server_key: "a26cd3a603139b2ec2bdb471060d78e5"
server_key_iv: "8d11e10c78ca09c4dbc7dff849ad08c9"
server_key_version: "encryptionkey@0"
I20260806 15:59:59.620563 3444 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 15:59:59.622812 3458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:59.623927 3444 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:59.624070 3444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "de2bd7c7aa524dfd8da27ee5da854e65"
format_stamp: "Formatted at 2026-08-06 15:59:59 on dist-test-slave-tm1g"
server_key: "a26cd3a603139b2ec2bdb471060d78e5"
server_key_iv: "8d11e10c78ca09c4dbc7dff849ad08c9"
server_key_version: "encryptionkey@0"
I20260806 15:59:59.624178 3444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:59.628662 3444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:59.629223 3444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:59.629406 3444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:59.635998 3444 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:46007
I20260806 15:59:59.636008 3513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:46007 every 8 connection(s)
I20260806 15:59:59.636839 3444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 15:59:59.639987 3514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:59:59.645362 3514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65: Bootstrap starting.
I20260806 15:59:59.645570 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 3444
I20260806 15:59:59.645696 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 15:59:59.645994 25726 external_mini_cluster.cc:1428] Setting key 8846f98c2939b104e8979e5b2c2752cf
I20260806 15:59:59.648010 3514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65: Neither blocks nor log segments found. Creating new log.
I20260806 15:59:59.648754 3514 log.cc:824] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65: Log is configured to *not* fsync() on all Append() calls
I20260806 15:59:59.651001 3514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65: No bootstrap required, opened a new log
I20260806 15:59:59.651602 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:59.647343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42860 (local address 127.25.31.190:46007)
0806 15:59:59.647972 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:59.647986 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:59.648019 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:59:59.648677 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:59.648692 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:59.649069 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:59.649456 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:59.650235 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:59.650240 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:59.650251 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:59.650260 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:59.650398 (+ 138us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:59.650428 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:59.650515 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:59.650655 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 15:59:59.650907 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":135,"threads_started":1}
I20260806 15:59:59.654428 3514 raft_consensus.cc:348] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2bd7c7aa524dfd8da27ee5da854e65" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 46007 } }
I20260806 15:59:59.654639 3514 raft_consensus.cc:374] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:59:59.654722 3514 raft_consensus.cc:729] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2bd7c7aa524dfd8da27ee5da854e65, State: Initialized, Role: FOLLOWER
I20260806 15:59:59.655125 3514 consensus_queue.cc:260] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2bd7c7aa524dfd8da27ee5da854e65" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 46007 } }
I20260806 15:59:59.655264 3514 raft_consensus.cc:388] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:59:59.655330 3514 raft_consensus.cc:482] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:59:59.655454 3514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:59:59.656280 3514 raft_consensus.cc:504] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2bd7c7aa524dfd8da27ee5da854e65" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 46007 } }
I20260806 15:59:59.656608 3514 leader_election.cc:302] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de2bd7c7aa524dfd8da27ee5da854e65; no voters:
I20260806 15:59:59.656888 3514 leader_election.cc:288] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:59:59.657021 3519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:59:59.657271 3519 raft_consensus.cc:686] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [term 1 LEADER]: Becoming Leader. State: Replica: de2bd7c7aa524dfd8da27ee5da854e65, State: Running, Role: LEADER
I20260806 15:59:59.657578 3519 consensus_queue.cc:237] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2bd7c7aa524dfd8da27ee5da854e65" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 46007 } }
I20260806 15:59:59.658074 3514 sys_catalog.cc:564] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:59:59.659297 3521 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de2bd7c7aa524dfd8da27ee5da854e65. Latest consensus state: current_term: 1 leader_uuid: "de2bd7c7aa524dfd8da27ee5da854e65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2bd7c7aa524dfd8da27ee5da854e65" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 46007 } } }
I20260806 15:59:59.659436 3521 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:59.659647 3520 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de2bd7c7aa524dfd8da27ee5da854e65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2bd7c7aa524dfd8da27ee5da854e65" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 46007 } } }
I20260806 15:59:59.659793 3520 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65 [sys.catalog]: This master's current role is: LEADER
I20260806 15:59:59.659837 3528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:59:59.662901 3528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:59:59.668193 3528 catalog_manager.cc:1347] Generated new cluster ID: 54f62648e2474362bad2dc8cbd2291eb
I20260806 15:59:59.668264 3528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:59:59.681772 3528 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:59:59.682070 3528 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:59:59.687415 3528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de2bd7c7aa524dfd8da27ee5da854e65: Generated new TSK 0
I20260806 15:59:59.687987 3528 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:59:59.698444 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:46007
--builtin_ntp_servers=127.25.31.148:44089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 15:59:59.815850 3538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:59.816093 3538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:59.816133 3538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:59.820111 3538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:59.820160 3538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:59.820255 3538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 15:59:59.825063 3538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:46007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:59.826135 3538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:59.827685 3538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:59:59.837277 3538 server_base.cc:1034] running on GCE node
W20260806 15:59:59.837226 3543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:59.837235 3546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:59:59.837409 3544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:59:59.837864 3538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:59:59.838500 3538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:59:59.839670 3538 hybrid_clock.cc:648] HybridClock initialized: now 1786031999839656 us; error 25 us; skew 500 ppm
I20260806 15:59:59.841480 3538 webserver.cc:460] Webserver started at http://127.25.31.129:32829/ using document root <none> and password file <none>
I20260806 15:59:59.842032 3538 fs_manager.cc:359] Metadata directory not provided
I20260806 15:59:59.842093 3538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:59:59.842387 3538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:59:59.844396 3538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "bcbf8ba7747641c08ab47d73e0cfa419"
format_stamp: "Formatted at 2026-08-06 15:59:59 on dist-test-slave-tm1g"
server_key: "bc443526a2b46b43d9273c1b3e5f694b"
server_key_iv: "f040cec8d70cfcfb9ac56df1e47e190b"
server_key_version: "encryptionkey@0"
I20260806 15:59:59.844952 3538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "bcbf8ba7747641c08ab47d73e0cfa419"
format_stamp: "Formatted at 2026-08-06 15:59:59 on dist-test-slave-tm1g"
server_key: "bc443526a2b46b43d9273c1b3e5f694b"
server_key_iv: "f040cec8d70cfcfb9ac56df1e47e190b"
server_key_version: "encryptionkey@0"
I20260806 15:59:59.848532 3538 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 15:59:59.850932 3552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:59.852090 3538 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 15:59:59.852217 3538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "bcbf8ba7747641c08ab47d73e0cfa419"
format_stamp: "Formatted at 2026-08-06 15:59:59 on dist-test-slave-tm1g"
server_key: "bc443526a2b46b43d9273c1b3e5f694b"
server_key_iv: "f040cec8d70cfcfb9ac56df1e47e190b"
server_key_version: "encryptionkey@0"
I20260806 15:59:59.852342 3538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:59:59.856736 3538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:59:59.857319 3538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:59:59.857491 3538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:59:59.858004 3538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:59:59.858959 3538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:59:59.859028 3538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:59.859091 3538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:59:59.859133 3538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:59:59.868137 3538 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:42229
I20260806 15:59:59.868161 3671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:42229 every 8 connection(s)
I20260806 15:59:59.868990 3538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 15:59:59.874882 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 3538
I20260806 15:59:59.874979 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:59.870801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:60857 (local address 127.25.31.190:46007)
0806 15:59:59.870943 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 15:59:59.870946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:59:59.871944 (+ 998us) server_negotiation.cc:407] Connection header received
0806 15:59:59.873288 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:59:59.873291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:59:59.873337 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:59:59.873421 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:59:59.874492 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:59:59.874495 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:59:59.874498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:59:59.874511 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:59:59.874544 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:59:59.874558 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:59:59.874658 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 15:59:59.874796 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 15:59:59.874843 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 15:59:59.875031 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 15:59:59.875342 25726 external_mini_cluster.cc:1428] Setting key 966e1f0c889e4169f30d163114754361
I20260806 15:59:59.875923 3674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:59:59.871174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:46007 (local address 127.25.31.129:60857)
0806 15:59:59.871804 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0806 15:59:59.871843 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:59:59.873080 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:59:59.873589 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:59:59.873596 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:59:59.873680 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:59:59.874198 (+ 518us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:59:59.874199 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:59:59.874236 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:59:59.874341 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:59:59.874643 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:59:59.874652 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:59:59.874658 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 15:59:59.874839 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 15:59:59.875071 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":109,"threads_started":1}
I20260806 15:59:59.877249 3672 heartbeater.cc:347] Connected to a master server at 127.25.31.190:46007
I20260806 15:59:59.877465 3672 heartbeater.cc:464] Registering TS with master...
I20260806 15:59:59.877497 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:46007
--builtin_ntp_servers=127.25.31.148:44089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 15:59:59.877908 3672 heartbeater.cc:511] Master 127.25.31.190:46007 requested a full tablet report, sending...
I20260806 15:59:59.879376 3477 ts_manager.cc:194] Registered new tserver with Master: bcbf8ba7747641c08ab47d73e0cfa419 (127.25.31.129:42229)
W20260806 15:59:59.987267 3675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:59:59.987499 3675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:59:59.987536 3675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:59:59.991508 3675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:59:59.991557 3675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:59:59.991611 3675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 15:59:59.996228 3675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:46007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 15:59:59.997206 3675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:59:59.998589 3675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:00:00.008129 3681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:00:00.008219 3675 server_base.cc:1034] running on GCE node
W20260806 16:00:00.008163 3680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:00.008149 3683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:00:00.008880 3675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:00:00.009503 3675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:00:00.010677 3675 hybrid_clock.cc:648] HybridClock initialized: now 1786032000010663 us; error 25 us; skew 500 ppm
I20260806 16:00:00.012562 3675 webserver.cc:460] Webserver started at http://127.25.31.130:40629/ using document root <none> and password file <none>
I20260806 16:00:00.013151 3675 fs_manager.cc:359] Metadata directory not provided
I20260806 16:00:00.013267 3675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:00:00.013464 3675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:00:00.015515 3675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "27f6434de8ae4e799c3849eeccb21399"
format_stamp: "Formatted at 2026-08-06 16:00:00 on dist-test-slave-tm1g"
server_key: "636d2cfc2d9e3fa7f07a8b56624d152f"
server_key_iv: "9e14f92e1757dfa94a190a76b4eddbe9"
server_key_version: "encryptionkey@0"
I20260806 16:00:00.016036 3675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "27f6434de8ae4e799c3849eeccb21399"
format_stamp: "Formatted at 2026-08-06 16:00:00 on dist-test-slave-tm1g"
server_key: "636d2cfc2d9e3fa7f07a8b56624d152f"
server_key_iv: "9e14f92e1757dfa94a190a76b4eddbe9"
server_key_version: "encryptionkey@0"
I20260806 16:00:00.019716 3675 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:00:00.022125 3689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:00.023370 3675 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:00:00.023512 3675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "27f6434de8ae4e799c3849eeccb21399"
format_stamp: "Formatted at 2026-08-06 16:00:00 on dist-test-slave-tm1g"
server_key: "636d2cfc2d9e3fa7f07a8b56624d152f"
server_key_iv: "9e14f92e1757dfa94a190a76b4eddbe9"
server_key_version: "encryptionkey@0"
I20260806 16:00:00.023614 3675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:00:00.028213 3675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:00:00.028833 3675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:00:00.029016 3675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:00:00.029542 3675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:00:00.030444 3675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:00:00.030493 3675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:00.030552 3675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:00:00.030591 3675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:00.039187 3675 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:36971
I20260806 16:00:00.039215 3808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:36971 every 8 connection(s)
I20260806 16:00:00.040030 3675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:00:00.043874 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 3675
I20260806 16:00:00.043994 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 16:00:00.044288 25726 external_mini_cluster.cc:1428] Setting key 494706d607b4158dda50a17c48673f05
I20260806 16:00:00.046113 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:00.041748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:38849 (local address 127.25.31.190:46007)
0806 16:00:00.041901 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:00.041904 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:00.042860 (+ 956us) server_negotiation.cc:407] Connection header received
0806 16:00:00.044245 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:00.044248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:00.044295 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:00.044403 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:00:00.045615 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:00:00.045619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:00:00.045622 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:00:00.045636 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:00:00.045664 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:00:00.045677 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:00:00.045768 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:00.045941 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 16:00:00.045984 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 16:00:00.046561 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:46007
--builtin_ntp_servers=127.25.31.148:44089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 16:00:00.047026 3811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:00.042056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:46007 (local address 127.25.31.130:38849)
0806 16:00:00.042720 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:00.042759 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:00.044018 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:00.044569 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:00.044575 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:00.044689 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:00:00.045284 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:00:00.045286 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:00:00.045353 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:00:00.045435 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:00:00.045770 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:00:00.045782 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:00:00.045788 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 16:00:00.045971 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 16:00:00.046184 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":105,"threads_started":1}
I20260806 16:00:00.048703 3809 heartbeater.cc:347] Connected to a master server at 127.25.31.190:46007
I20260806 16:00:00.048929 3809 heartbeater.cc:464] Registering TS with master...
I20260806 16:00:00.049376 3809 heartbeater.cc:511] Master 127.25.31.190:46007 requested a full tablet report, sending...
I20260806 16:00:00.050443 3477 ts_manager.cc:194] Registered new tserver with Master: 27f6434de8ae4e799c3849eeccb21399 (127.25.31.130:36971)
W20260806 16:00:00.158483 3812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:00:00.158708 3812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:00:00.158775 3812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:00:00.162938 3812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:00:00.162992 3812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:00:00.163069 3812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:00:00.167932 3812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:46007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:00:00.168919 3812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:00:00.170225 3812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:00:00.179685 3818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:00.179664 3817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:00.179663 3820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:00:00.179700 3812 server_base.cc:1034] running on GCE node
I20260806 16:00:00.180336 3812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:00:00.180932 3812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:00:00.182123 3812 hybrid_clock.cc:648] HybridClock initialized: now 1786032000182116 us; error 28 us; skew 500 ppm
I20260806 16:00:00.183919 3812 webserver.cc:460] Webserver started at http://127.25.31.131:42665/ using document root <none> and password file <none>
I20260806 16:00:00.184499 3812 fs_manager.cc:359] Metadata directory not provided
I20260806 16:00:00.184594 3812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:00:00.184811 3812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:00:00.186995 3812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "a5254b48fbca48c2809db1d57c69fdaf"
format_stamp: "Formatted at 2026-08-06 16:00:00 on dist-test-slave-tm1g"
server_key: "9c08973b777326200d69fb8559ad2075"
server_key_iv: "0e057709b99edb13d2eb05f6ed570a38"
server_key_version: "encryptionkey@0"
I20260806 16:00:00.187669 3812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "a5254b48fbca48c2809db1d57c69fdaf"
format_stamp: "Formatted at 2026-08-06 16:00:00 on dist-test-slave-tm1g"
server_key: "9c08973b777326200d69fb8559ad2075"
server_key_iv: "0e057709b99edb13d2eb05f6ed570a38"
server_key_version: "encryptionkey@0"
I20260806 16:00:00.191267 3812 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:00:00.193684 3826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:00.194900 3812 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:00:00.195044 3812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "a5254b48fbca48c2809db1d57c69fdaf"
format_stamp: "Formatted at 2026-08-06 16:00:00 on dist-test-slave-tm1g"
server_key: "9c08973b777326200d69fb8559ad2075"
server_key_iv: "0e057709b99edb13d2eb05f6ed570a38"
server_key_version: "encryptionkey@0"
I20260806 16:00:00.195161 3812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:00:00.199502 3812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:00:00.200093 3812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:00:00.200274 3812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:00:00.200794 3812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:00:00.201666 3812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:00:00.201736 3812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:00.201804 3812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:00:00.201840 3812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:00.210712 3812 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:45923
I20260806 16:00:00.210733 3943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:45923 every 8 connection(s)
I20260806 16:00:00.211584 3812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:00:00.213163 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 3812
I20260806 16:00:00.213289 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 16:00:00.213647 25726 external_mini_cluster.cc:1428] Setting key b622bd115d590c0a2743d1af73870a5f
I20260806 16:00:00.217667 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:00.213358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:48925 (local address 127.25.31.190:46007)
0806 16:00:00.213538 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:00.213545 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:00.214770 (+ 1225us) server_negotiation.cc:407] Connection header received
0806 16:00:00.216283 (+ 1513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:00.216285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:00.216320 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:00.216387 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:00:00.217341 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:00:00.217344 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:00:00.217346 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:00:00.217356 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:00:00.217378 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:00:00.217385 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:00:00.217454 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:00.217527 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 16:00:00.217568 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 16:00:00.218519 3946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:00.213778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:46007 (local address 127.25.31.131:48925)
0806 16:00:00.214612 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:00.214668 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:00.216104 (+ 1436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:00.216507 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:00.216522 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:00.216602 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:00:00.217109 (+ 507us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:00:00.217110 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:00:00.217149 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:00:00.217212 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:00:00.217441 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:00:00.217446 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:00:00.217449 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 16:00:00.217563 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 16:00:00.217774 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":163,"threads_started":1}
I20260806 16:00:00.219612 3944 heartbeater.cc:347] Connected to a master server at 127.25.31.190:46007
I20260806 16:00:00.219828 3944 heartbeater.cc:464] Registering TS with master...
I20260806 16:00:00.220208 3944 heartbeater.cc:511] Master 127.25.31.190:46007 requested a full tablet report, sending...
I20260806 16:00:00.221156 3477 ts_manager.cc:194] Registered new tserver with Master: a5254b48fbca48c2809db1d57c69fdaf (127.25.31.131:45923)
I20260806 16:00:00.227319 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 16:00:00.230135 3517 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:00:00.229261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42868 (local address 127.25.31.190:46007)
0806 16:00:00.229405 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:00.229410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:00.229619 (+ 209us) server_negotiation.cc:407] Connection header received
0806 16:00:00.229693 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:00.229695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:00.229725 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:00.229788 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:00:00.230019 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42868: BlockingRecv error: recv got EOF from 127.0.0.1:42868 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 16:00:00.230288 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 3538
I20260806 16:00:00.239276 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 3675
I20260806 16:00:00.245786 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 3812
I20260806 16:00:00.251335 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 3444
2026-08-06T16:00:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (800 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:00:00 dist-test-slave-tm1g krb5kdc[3954](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:00:00 dist-test-slave-tm1g krb5kdc[3954](info): set up 2 sockets
Aug 06 16:00:00 dist-test-slave-tm1g krb5kdc[3954](info): commencing operation
krb5kdc: starting...
W20260806 16:00:01.308151 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:00:01 dist-test-slave-tm1g krb5kdc[3954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:00:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:00:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:00:01.429522 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:35621
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:35621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:00:01.539455 3967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:00:01.539744 3967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:00:01.539817 3967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:00:01.543836 3967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:00:01.543912 3967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:00:01.543963 3967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:00:01.544021 3967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:00:01.544066 3967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:00:01.550206 3967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:35621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:35621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:00:01.551575 3967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:00:01.552963 3967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:00:01.562216 3972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:01.562276 3973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:01.562220 3975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:00:01.562579 3967 server_base.cc:1034] running on GCE node
I20260806 16:00:01.563162 3967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:00:01.564225 3967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:00:01.565385 3967 hybrid_clock.cc:648] HybridClock initialized: now 1786032001565356 us; error 41 us; skew 500 ppm
Aug 06 16:00:01 dist-test-slave-tm1g krb5kdc[3954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032001, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:00:01.568181 3967 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:00:01.568970 3967 webserver.cc:460] Webserver started at http://127.25.31.190:40977/ using document root <none> and password file <none>
I20260806 16:00:01.569469 3967 fs_manager.cc:359] Metadata directory not provided
I20260806 16:00:01.569535 3967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:00:01.569693 3967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:00:01.571712 3967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "9e24ebe83f824fce9e6917f1560599fc"
format_stamp: "Formatted at 2026-08-06 16:00:01 on dist-test-slave-tm1g"
server_key: "347e1951ea7cbd5f6f2ac7829cc0fad9"
server_key_iv: "859942c7fb0eb6cafe827724d95b32e2"
server_key_version: "encryptionkey@0"
I20260806 16:00:01.572310 3967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "9e24ebe83f824fce9e6917f1560599fc"
format_stamp: "Formatted at 2026-08-06 16:00:01 on dist-test-slave-tm1g"
server_key: "347e1951ea7cbd5f6f2ac7829cc0fad9"
server_key_iv: "859942c7fb0eb6cafe827724d95b32e2"
server_key_version: "encryptionkey@0"
I20260806 16:00:01.576077 3967 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 16:00:01.578467 3982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:01.579667 3967 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:00:01.579809 3967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "9e24ebe83f824fce9e6917f1560599fc"
format_stamp: "Formatted at 2026-08-06 16:00:01 on dist-test-slave-tm1g"
server_key: "347e1951ea7cbd5f6f2ac7829cc0fad9"
server_key_iv: "859942c7fb0eb6cafe827724d95b32e2"
server_key_version: "encryptionkey@0"
I20260806 16:00:01.579918 3967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:00:01.602604 3967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:00:01.606413 3967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:00:01.606624 3967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:00:01.613891 3967 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:35621
I20260806 16:00:01.613925 4035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:35621 every 8 connection(s)
I20260806 16:00:01.614934 3967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:00:01.615760 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 3967
I20260806 16:00:01.615880 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:00:01.616190 25726 external_mini_cluster.cc:1428] Setting key 1e54337bc05697754500eda8b6ead0f3
I20260806 16:00:01.618206 4036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:00:01.624848 4036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc: Bootstrap starting.
Aug 06 16:00:01 dist-test-slave-tm1g krb5kdc[3954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:00:01.627626 4036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc: Neither blocks nor log segments found. Creating new log.
I20260806 16:00:01.628588 4036 log.cc:824] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc: Log is configured to *not* fsync() on all Append() calls
I20260806 16:00:01.630901 4036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc: No bootstrap required, opened a new log
I20260806 16:00:01.631098 4039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:01.617538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45374 (local address 127.25.31.190:35621)
0806 16:00:01.617948 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:01.617964 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:01.618000 (+ 36us) server_negotiation.cc:407] Connection header received
0806 16:00:01.618736 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:01.618754 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:01.619089 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:01.619360 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:00:01.620258 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:01.621254 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:01.622418 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:01.622873 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:01.622999 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:01.625810 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:00:01.625816 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:00:01.625832 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:00:01.625872 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:00:01.627958 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:01.628498 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:01.628504 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:01.628510 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:01.628658 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:01.629001 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:01.629007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:01.629012 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:01.629447 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:00:01.629651 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:01.629864 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 16:00:01.630162 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":144,"threads_started":1}
I20260806 16:00:01.634742 4036 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e24ebe83f824fce9e6917f1560599fc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35621 } }
I20260806 16:00:01.634949 4036 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:00:01.634984 4036 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e24ebe83f824fce9e6917f1560599fc, State: Initialized, Role: FOLLOWER
I20260806 16:00:01.635380 4036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e24ebe83f824fce9e6917f1560599fc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35621 } }
I20260806 16:00:01.635502 4036 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:00:01.635555 4036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:00:01.635627 4036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:00:01.636417 4036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e24ebe83f824fce9e6917f1560599fc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35621 } }
I20260806 16:00:01.636708 4036 leader_election.cc:302] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e24ebe83f824fce9e6917f1560599fc; no voters:
I20260806 16:00:01.636956 4036 leader_election.cc:288] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:00:01.637084 4041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:00:01.637616 4041 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [term 1 LEADER]: Becoming Leader. State: Replica: 9e24ebe83f824fce9e6917f1560599fc, State: Running, Role: LEADER
I20260806 16:00:01.637948 4041 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e24ebe83f824fce9e6917f1560599fc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35621 } }
I20260806 16:00:01.638221 4036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:00:01.639607 4042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e24ebe83f824fce9e6917f1560599fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e24ebe83f824fce9e6917f1560599fc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35621 } } }
I20260806 16:00:01.639963 4043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e24ebe83f824fce9e6917f1560599fc. Latest consensus state: current_term: 1 leader_uuid: "9e24ebe83f824fce9e6917f1560599fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e24ebe83f824fce9e6917f1560599fc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35621 } } }
I20260806 16:00:01.640077 4043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [sys.catalog]: This master's current role is: LEADER
I20260806 16:00:01.639735 4042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc [sys.catalog]: This master's current role is: LEADER
I20260806 16:00:01.640697 4055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:00:01.643286 4055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:00:01.648629 4055 catalog_manager.cc:1347] Generated new cluster ID: 4ac7cbbced0a46cab8b85e04da67a7c9
I20260806 16:00:01.648693 4055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:00:01.662575 4055 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:00:01.663431 4055 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:00:01.669450 4055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc: Generated new TSK 0
I20260806 16:00:01.670203 4055 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 16:00:01.699745 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35621
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:00:01.817865 4063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:00:01.818162 4063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:00:01.818256 4063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:00:01.822377 4063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:00:01.822453 4063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:00:01.822544 4063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:00:01.827836 4063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:00:01.828881 4063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:00:01.829895 4063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:00:01.839718 4063 server_base.cc:1034] running on GCE node
W20260806 16:00:01.839670 4072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:01.839740 4070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:01.839681 4069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:00:01.840286 4063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:00:01.840876 4063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:00:01.842041 4063 hybrid_clock.cc:648] HybridClock initialized: now 1786032001842031 us; error 29 us; skew 500 ppm
Aug 06 16:00:01 dist-test-slave-tm1g krb5kdc[3954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032001, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:00:01.844696 4063 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 16:00:01.845535 4063 webserver.cc:460] Webserver started at http://127.25.31.129:41321/ using document root <none> and password file <none>
I20260806 16:00:01.846076 4063 fs_manager.cc:359] Metadata directory not provided
I20260806 16:00:01.846166 4063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:00:01.846457 4063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:00:01.848416 4063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "b34790b85baa4d52a1f3335bcf6164d2"
format_stamp: "Formatted at 2026-08-06 16:00:01 on dist-test-slave-tm1g"
server_key: "2ec44d5de0bee9f8c8e2547e2e827680"
server_key_iv: "006f9cbc522dba7ff423b7218eb3ea2d"
server_key_version: "encryptionkey@0"
I20260806 16:00:01.848985 4063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "b34790b85baa4d52a1f3335bcf6164d2"
format_stamp: "Formatted at 2026-08-06 16:00:01 on dist-test-slave-tm1g"
server_key: "2ec44d5de0bee9f8c8e2547e2e827680"
server_key_iv: "006f9cbc522dba7ff423b7218eb3ea2d"
server_key_version: "encryptionkey@0"
I20260806 16:00:01.852555 4063 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 16:00:01.854884 4079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:01.856029 4063 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:00:01.856158 4063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "b34790b85baa4d52a1f3335bcf6164d2"
format_stamp: "Formatted at 2026-08-06 16:00:01 on dist-test-slave-tm1g"
server_key: "2ec44d5de0bee9f8c8e2547e2e827680"
server_key_iv: "006f9cbc522dba7ff423b7218eb3ea2d"
server_key_version: "encryptionkey@0"
I20260806 16:00:01.856263 4063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:00:01.873329 4063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:00:01.875939 4063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:00:01.876134 4063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:00:01.876683 4063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:00:01.877449 4063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:00:01.877516 4063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:01.877585 4063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:00:01.877638 4063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:01.886510 4063 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:36121
I20260806 16:00:01.886560 4193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:36121 every 8 connection(s)
I20260806 16:00:01.887384 4063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
Aug 06 16:00:01 dist-test-slave-tm1g krb5kdc[3954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032001, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:00:01.896436 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 4063
I20260806 16:00:01.896567 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 16:00:01.896857 25726 external_mini_cluster.cc:1428] Setting key 04ee6777ca94c3d2e2c87e5404a85caa
I20260806 16:00:01.901021 4039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:01.889126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:60331 (local address 127.25.31.190:35621)
0806 16:00:01.889288 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:01.889294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:01.890162 (+ 868us) server_negotiation.cc:407] Connection header received
0806 16:00:01.891530 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:01.891533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:01.891590 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:01.891663 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:00:01.892708 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:01.893237 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:01.894332 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:01.894572 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:01.894650 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:01.897077 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:00:01.897080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:00:01.897082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:00:01.897105 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:00:01.898866 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:01.899372 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:01.899378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:01.899383 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:01.899478 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:01.899793 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:01.899799 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:01.899803 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:01.900064 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:00:01.900177 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:01.900797 (+ 620us) server_negotiation.cc:299] Negotiation successful
0806 16:00:01.900906 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 16:00:01.902017 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:01.889573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:35621 (local address 127.25.31.129:60331)
0806 16:00:01.890007 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:01.890047 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:01.891324 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:01.891785 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:01.891791 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:01.892119 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:00:01.892580 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:01.892593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:01.893344 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:01.893347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:01.894209 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:01.894218 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:01.894705 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:01.894708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:01.894854 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:01.895369 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:00:01.895383 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:00:01.896892 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:00:01.898986 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:00:01.898991 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:00:01.899001 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:00:01.899271 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:00:01.899570 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:00:01.899572 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:00:01.899574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:00:01.899701 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:00:01.900196 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:00:01.900205 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:00:01.900526 (+ 321us) client_negotiation.cc:769] Sending connection context
0806 16:00:01.900759 (+ 233us) client_negotiation.cc:240] Negotiation successful
0806 16:00:01.901118 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":116,"threads_started":1}
I20260806 16:00:01.903367 4194 heartbeater.cc:347] Connected to a master server at 127.25.31.190:35621
I20260806 16:00:01.903605 4194 heartbeater.cc:464] Registering TS with master...
I20260806 16:00:01.904072 4194 heartbeater.cc:511] Master 127.25.31.190:35621 requested a full tablet report, sending...
I20260806 16:00:01.905578 3999 ts_manager.cc:194] Registered new tserver with Master: b34790b85baa4d52a1f3335bcf6164d2 (127.25.31.129:36121)
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
I20260806 16:00:01.906746 3999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:60331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 16:00:01.925509 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35621
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:00:02.034355 4200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:00:02.034592 4200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:00:02.034634 4200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:00:02.038980 4200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:00:02.039031 4200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:00:02.039088 4200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:00:02.044240 4200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:00:02.045138 4200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:00:02.046090 4200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:00:02.055716 4206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:00:02.055827 4200 server_base.cc:1034] running on GCE node
W20260806 16:00:02.055721 4205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:02.055755 4208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:00:02.056506 4200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:00:02.057098 4200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:00:02.058287 4200 hybrid_clock.cc:648] HybridClock initialized: now 1786032002058261 us; error 37 us; skew 500 ppm
Aug 06 16:00:02 dist-test-slave-tm1g krb5kdc[3954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032002, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:00:02.061021 4200 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 16:00:02.061856 4200 webserver.cc:460] Webserver started at http://127.25.31.130:40169/ using document root <none> and password file <none>
I20260806 16:00:02.062458 4200 fs_manager.cc:359] Metadata directory not provided
I20260806 16:00:02.062548 4200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:00:02.062772 4200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:00:02.064700 4200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "4f79cbff6394473a9515069cd9a8714e"
format_stamp: "Formatted at 2026-08-06 16:00:02 on dist-test-slave-tm1g"
server_key: "2dbef3d02f5beda1fc573f92b48bb273"
server_key_iv: "c90a00ce2f16fc4107ed37b5c67f3326"
server_key_version: "encryptionkey@0"
I20260806 16:00:02.065284 4200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "4f79cbff6394473a9515069cd9a8714e"
format_stamp: "Formatted at 2026-08-06 16:00:02 on dist-test-slave-tm1g"
server_key: "2dbef3d02f5beda1fc573f92b48bb273"
server_key_iv: "c90a00ce2f16fc4107ed37b5c67f3326"
server_key_version: "encryptionkey@0"
I20260806 16:00:02.068825 4200 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 16:00:02.071234 4216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:02.072350 4200 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:00:02.072484 4200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "4f79cbff6394473a9515069cd9a8714e"
format_stamp: "Formatted at 2026-08-06 16:00:02 on dist-test-slave-tm1g"
server_key: "2dbef3d02f5beda1fc573f92b48bb273"
server_key_iv: "c90a00ce2f16fc4107ed37b5c67f3326"
server_key_version: "encryptionkey@0"
I20260806 16:00:02.072595 4200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:00:02.084539 4200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:00:02.087235 4200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:00:02.087431 4200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:00:02.087986 4200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:00:02.088755 4200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:00:02.088825 4200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:02.088907 4200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:00:02.088943 4200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:02.097798 4200 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:37777
I20260806 16:00:02.097837 4331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:37777 every 8 connection(s)
I20260806 16:00:02.098698 4200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:00:02.101347 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 4200
I20260806 16:00:02.101500 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 16:00:02.101836 25726 external_mini_cluster.cc:1428] Setting key 0794d9fa0571c78bd67d15b89ea19859
Aug 06 16:00:02 dist-test-slave-tm1g krb5kdc[3954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032002, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
I20260806 16:00:02.113763 4039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.100455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:54909 (local address 127.25.31.190:35621)
0806 16:00:02.100600 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:02.100604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:02.101537 (+ 933us) server_negotiation.cc:407] Connection header received
0806 16:00:02.103324 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:02.103327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:02.103387 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:02.103464 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:00:02.104558 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.105398 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.107005 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.107269 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.107402 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.109960 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:00:02.109966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:00:02.109973 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:00:02.110007 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:00:02.111756 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:02.112259 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:02.112264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:02.112267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:02.112362 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:02.112693 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:02.112696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:02.112698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:02.112925 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:00:02.113028 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:02.113544 (+ 516us) server_negotiation.cc:299] Negotiation successful
0806 16:00:02.113645 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 16:00:02.114667 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.100820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:35621 (local address 127.25.31.130:54909)
0806 16:00:02.101366 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:02.101427 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:02.103114 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:02.103621 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:02.103628 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:02.103970 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:00:02.104419 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:02.104432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.105602 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.105607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:02.106888 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:02.106899 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.107428 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.107433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:02.107616 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.108230 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:00:02.108246 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:00:02.109780 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:00:02.111872 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:00:02.111876 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:00:02.111888 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:00:02.112150 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:00:02.112466 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:00:02.112468 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:00:02.112469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:00:02.112599 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:00:02.113030 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:00:02.113034 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:00:02.113303 (+ 269us) client_negotiation.cc:769] Sending connection context
0806 16:00:02.113500 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 16:00:02.113752 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":116,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
I20260806 16:00:02.115900 4332 heartbeater.cc:347] Connected to a master server at 127.25.31.190:35621
I20260806 16:00:02.116137 4332 heartbeater.cc:464] Registering TS with master...
I20260806 16:00:02.116737 4332 heartbeater.cc:511] Master 127.25.31.190:35621 requested a full tablet report, sending...
I20260806 16:00:02.117882 3999 ts_manager.cc:194] Registered new tserver with Master: 4f79cbff6394473a9515069cd9a8714e (127.25.31.130:37777)
I20260806 16:00:02.118521 3999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:54909
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 16:00:02.133673 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35621
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:35597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:00:02.246151 4337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:00:02.246423 4337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:00:02.246466 4337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:00:02.250536 4337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:00:02.250590 4337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:00:02.250646 4337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:00:02.255941 4337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:35597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:00:02.256839 4337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:00:02.257768 4337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:00:02.267460 4345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:02.267503 4342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:00:02.267575 4337 server_base.cc:1034] running on GCE node
W20260806 16:00:02.267459 4343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:00:02.268183 4337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:00:02.268844 4337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:00:02.270025 4337 hybrid_clock.cc:648] HybridClock initialized: now 1786032002270007 us; error 28 us; skew 500 ppm
Aug 06 16:00:02 dist-test-slave-tm1g krb5kdc[3954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032002, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:00:02.272737 4337 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 16:00:02.273561 4337 webserver.cc:460] Webserver started at http://127.25.31.131:45061/ using document root <none> and password file <none>
I20260806 16:00:02.274123 4337 fs_manager.cc:359] Metadata directory not provided
I20260806 16:00:02.274217 4337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:00:02.274469 4337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:00:02.276494 4337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "940e4a072eb547939a7edd6fbb694754"
format_stamp: "Formatted at 2026-08-06 16:00:02 on dist-test-slave-tm1g"
server_key: "8464ed9ca996d79f9b56e074c0c6653c"
server_key_iv: "ea771f806da5c826bb93653651e40642"
server_key_version: "encryptionkey@0"
I20260806 16:00:02.277089 4337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "940e4a072eb547939a7edd6fbb694754"
format_stamp: "Formatted at 2026-08-06 16:00:02 on dist-test-slave-tm1g"
server_key: "8464ed9ca996d79f9b56e074c0c6653c"
server_key_iv: "ea771f806da5c826bb93653651e40642"
server_key_version: "encryptionkey@0"
I20260806 16:00:02.280789 4337 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 16:00:02.283236 4353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:02.284454 4337 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:00:02.284588 4337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "940e4a072eb547939a7edd6fbb694754"
format_stamp: "Formatted at 2026-08-06 16:00:02 on dist-test-slave-tm1g"
server_key: "8464ed9ca996d79f9b56e074c0c6653c"
server_key_iv: "ea771f806da5c826bb93653651e40642"
server_key_version: "encryptionkey@0"
I20260806 16:00:02.284698 4337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:00:02.305055 4337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:00:02.308095 4337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:00:02.308300 4337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:00:02.308902 4337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:00:02.309806 4337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:00:02.309880 4337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:02.309953 4337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:00:02.310005 4337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:02.319264 4337 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:37769
I20260806 16:00:02.319289 4469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:37769 every 8 connection(s)
I20260806 16:00:02.320187 4337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
Aug 06 16:00:02 dist-test-slave-tm1g krb5kdc[3954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032002, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:00:02.329777 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 4337
I20260806 16:00:02.329916 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 16:00:02.330216 25726 external_mini_cluster.cc:1428] Setting key ae4ec7b683bcfdb5b17cca5eeaec4f16
I20260806 16:00:02.334204 4039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.322018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:41929 (local address 127.25.31.190:35621)
0806 16:00:02.322166 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:02.322169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:02.323017 (+ 848us) server_negotiation.cc:407] Connection header received
0806 16:00:02.324430 (+ 1413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:02.324434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:02.324490 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:02.324608 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:00:02.325818 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.326436 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.327589 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.327826 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.327939 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.330472 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:00:02.330476 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:00:02.330479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:00:02.330507 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:00:02.332248 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:02.332889 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:02.332892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:02.332893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:02.332988 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:02.333294 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:02.333296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:02.333298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:02.333472 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:00:02.333548 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:02.333992 (+ 444us) server_negotiation.cc:299] Negotiation successful
0806 16:00:02.334089 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 16:00:02.335031 4472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.322415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:35621 (local address 127.25.31.131:41929)
0806 16:00:02.322872 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:02.322914 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:02.324218 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:02.324808 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:02.324815 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:02.325173 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:00:02.325661 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:02.325675 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.326564 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.326568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:02.327459 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:02.327468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.327982 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.327986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:02.328166 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.328713 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:00:02.328728 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:00:02.330253 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:00:02.332389 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:00:02.332396 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:00:02.332413 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:00:02.332789 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:00:02.333075 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:00:02.333077 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:00:02.333079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:00:02.333213 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:00:02.333566 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:00:02.333571 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:00:02.333787 (+ 216us) client_negotiation.cc:769] Sending connection context
0806 16:00:02.333939 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 16:00:02.334179 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":113,"threads_started":1}
I20260806 16:00:02.336166 4470 heartbeater.cc:347] Connected to a master server at 127.25.31.190:35621
I20260806 16:00:02.336372 4470 heartbeater.cc:464] Registering TS with master...
I20260806 16:00:02.336799 4470 heartbeater.cc:511] Master 127.25.31.190:35621 requested a full tablet report, sending...
I20260806 16:00:02.337729 3999 ts_manager.cc:194] Registered new tserver with Master: 940e4a072eb547939a7edd6fbb694754 (127.25.31.131:37769)
I20260806 16:00:02.338267 3999 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:41929
I20260806 16:00:02.344149 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:00:02.352972 4039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.346076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45380 (local address 127.25.31.190:35621)
0806 16:00:02.346198 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:02.346202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:02.346431 (+ 229us) server_negotiation.cc:407] Connection header received
0806 16:00:02.346637 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:02.346640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:02.346691 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:02.346765 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:00:02.347410 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.347903 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.348682 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.348884 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.348955 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.350235 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:00:02.350238 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:00:02.350240 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:00:02.350261 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:00:02.351580 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:02.351965 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:02.351967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:02.351968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:02.352040 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:02.352264 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:02.352266 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:02.352267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:02.352488 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:00:02.352590 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:02.352773 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 16:00:02.352862 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 16:00:02.355739 3999 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:00:02.358758 3999 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:00:02.372853 4482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.367106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47502 (local address 127.25.31.129:36121)
0806 16:00:02.368253 (+ 1147us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:02.368268 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:02.368291 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:00:02.368377 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:02.368383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:02.368581 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:02.368743 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:00:02.369073 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.369893 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.371638 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.372179 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.372304 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.372465 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:02.372606 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 16:00:02.372700 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1025,"thread_start_us":73,"threads_started":1}
I20260806 16:00:02.372848 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.366971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:36121 (local address 127.0.0.1:47502)
0806 16:00:02.367574 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:02.367626 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:02.367752 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:02.368749 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:02.368754 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:02.368777 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:00:02.368943 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:02.368952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.370051 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.370055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:02.371541 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:02.371552 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.372307 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.372311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:02.372423 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.372462 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 16:00:02.372576 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 16:00:02.372692 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":82,"thread_start_us":66,"threads_started":1}
I20260806 16:00:02.373128 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.368158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59702 (local address 127.25.31.130:37777)
0806 16:00:02.368506 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:02.368516 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:02.368535 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:00:02.368595 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:02.368598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:02.368778 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:02.368982 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:00:02.369255 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.369965 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.371468 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.372385 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.372667 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.372813 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:02.372865 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 16:00:02.372972 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":142,"threads_started":1}
I20260806 16:00:02.373399 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.367366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:37777 (local address 127.0.0.1:59702)
0806 16:00:02.367981 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:02.367992 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:02.368071 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:02.368969 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:02.368972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:02.369004 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:00:02.369134 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:02.369141 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.370123 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.370127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:02.371347 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:02.371357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.372513 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.372517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:02.372606 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.372654 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 16:00:02.372715 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 16:00:02.372784 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":20,"spinlock_wait_cycles":6272,"thread_start_us":92,"threads_started":1}
I20260806 16:00:02.374600 4484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.367107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:37769 (local address 127.0.0.1:46192)
0806 16:00:02.367890 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:02.367905 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:02.367980 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:02.368969 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:02.368972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:02.369003 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:00:02.370115 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:02.370122 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.371569 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.371572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:02.372872 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:02.372881 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.374157 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.374160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:02.374247 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.374274 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 16:00:02.374400 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 16:00:02.374469 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":989,"spinlock_wait_cycles":5760,"thread_start_us":97,"threads_started":1}
I20260806 16:00:02.375001 4485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.367745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46192 (local address 127.25.31.131:37769)
0806 16:00:02.368306 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:02.368332 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:02.368350 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:00:02.368412 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:02.368415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:02.368581 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:02.368743 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:00:02.370251 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.371436 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.373000 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.373843 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.373943 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.374093 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:02.374786 (+ 693us) server_negotiation.cc:299] Negotiation successful
0806 16:00:02.374890 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":135,"threads_started":1}
I20260806 16:00:02.376269 4264 tablet_service.cc:1467] Processing CreateTablet for tablet 25a2ca471c8f48109c396ed5f2ffb532 (DEFAULT_TABLE table=test-table [id=87ea832f4e60484fb8bbc70275983796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:00:02.376268 4127 tablet_service.cc:1467] Processing CreateTablet for tablet 25a2ca471c8f48109c396ed5f2ffb532 (DEFAULT_TABLE table=test-table [id=87ea832f4e60484fb8bbc70275983796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:00:02.377404 4264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25a2ca471c8f48109c396ed5f2ffb532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:00:02.377404 4127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25a2ca471c8f48109c396ed5f2ffb532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:00:02.380401 4403 tablet_service.cc:1467] Processing CreateTablet for tablet 25a2ca471c8f48109c396ed5f2ffb532 (DEFAULT_TABLE table=test-table [id=87ea832f4e60484fb8bbc70275983796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:00:02.381239 4403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25a2ca471c8f48109c396ed5f2ffb532. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:00:02.385003 4488 tablet_bootstrap.cc:492] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e: Bootstrap starting.
I20260806 16:00:02.385257 4487 tablet_bootstrap.cc:492] T 25a2ca471c8f48109c396ed5f2ffb532 P b34790b85baa4d52a1f3335bcf6164d2: Bootstrap starting.
I20260806 16:00:02.388212 4487 tablet_bootstrap.cc:654] T 25a2ca471c8f48109c396ed5f2ffb532 P b34790b85baa4d52a1f3335bcf6164d2: Neither blocks nor log segments found. Creating new log.
I20260806 16:00:02.388281 4488 tablet_bootstrap.cc:654] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e: Neither blocks nor log segments found. Creating new log.
I20260806 16:00:02.388532 4489 tablet_bootstrap.cc:492] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754: Bootstrap starting.
I20260806 16:00:02.389197 4488 log.cc:824] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e: Log is configured to *not* fsync() on all Append() calls
I20260806 16:00:02.389796 4487 log.cc:824] T 25a2ca471c8f48109c396ed5f2ffb532 P b34790b85baa4d52a1f3335bcf6164d2: Log is configured to *not* fsync() on all Append() calls
I20260806 16:00:02.391083 4489 tablet_bootstrap.cc:654] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754: Neither blocks nor log segments found. Creating new log.
I20260806 16:00:02.391201 4488 tablet_bootstrap.cc:492] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e: No bootstrap required, opened a new log
I20260806 16:00:02.391410 4488 ts_tablet_manager.cc:1397] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260806 16:00:02.391822 4487 tablet_bootstrap.cc:492] T 25a2ca471c8f48109c396ed5f2ffb532 P b34790b85baa4d52a1f3335bcf6164d2: No bootstrap required, opened a new log
I20260806 16:00:02.391887 4489 log.cc:824] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754: Log is configured to *not* fsync() on all Append() calls
I20260806 16:00:02.392023 4487 ts_tablet_manager.cc:1397] T 25a2ca471c8f48109c396ed5f2ffb532 P b34790b85baa4d52a1f3335bcf6164d2: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 16:00:02.393817 4489 tablet_bootstrap.cc:492] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754: No bootstrap required, opened a new log
I20260806 16:00:02.394001 4489 ts_tablet_manager.cc:1397] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 16:00:02.394737 4488 raft_consensus.cc:348] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34790b85baa4d52a1f3335bcf6164d2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36121 } } peers { permanent_uuid: "940e4a072eb547939a7edd6fbb694754" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37769 } } peers { permanent_uuid: "4f79cbff6394473a9515069cd9a8714e" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 37777 } }
I20260806 16:00:02.395107 4488 raft_consensus.cc:374] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:00:02.395201 4488 raft_consensus.cc:729] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f79cbff6394473a9515069cd9a8714e, State: Initialized, Role: FOLLOWER
I20260806 16:00:02.395641 4488 consensus_queue.cc:260] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34790b85baa4d52a1f3335bcf6164d2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36121 } } peers { permanent_uuid: "940e4a072eb547939a7edd6fbb694754" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37769 } } peers { permanent_uuid: "4f79cbff6394473a9515069cd9a8714e" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 37777 } }
I20260806 16:00:02.395737 4487 raft_consensus.cc:348] T 25a2ca471c8f48109c396ed5f2ffb532 P b34790b85baa4d52a1f3335bcf6164d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34790b85baa4d52a1f3335bcf6164d2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36121 } } peers { permanent_uuid: "940e4a072eb547939a7edd6fbb694754" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37769 } } peers { permanent_uuid: "4f79cbff6394473a9515069cd9a8714e" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 37777 } }
I20260806 16:00:02.396059 4487 raft_consensus.cc:374] T 25a2ca471c8f48109c396ed5f2ffb532 P b34790b85baa4d52a1f3335bcf6164d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:00:02.396155 4487 raft_consensus.cc:729] T 25a2ca471c8f48109c396ed5f2ffb532 P b34790b85baa4d52a1f3335bcf6164d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b34790b85baa4d52a1f3335bcf6164d2, State: Initialized, Role: FOLLOWER
I20260806 16:00:02.396278 4332 heartbeater.cc:503] Master 127.25.31.190:35621 was elected leader, sending a full tablet report...
I20260806 16:00:02.396610 4488 ts_tablet_manager.cc:1428] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 16:00:02.396610 4487 consensus_queue.cc:260] T 25a2ca471c8f48109c396ed5f2ffb532 P b34790b85baa4d52a1f3335bcf6164d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34790b85baa4d52a1f3335bcf6164d2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36121 } } peers { permanent_uuid: "940e4a072eb547939a7edd6fbb694754" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37769 } } peers { permanent_uuid: "4f79cbff6394473a9515069cd9a8714e" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 37777 } }
I20260806 16:00:02.397240 4194 heartbeater.cc:503] Master 127.25.31.190:35621 was elected leader, sending a full tablet report...
I20260806 16:00:02.397537 4487 ts_tablet_manager.cc:1428] T 25a2ca471c8f48109c396ed5f2ffb532 P b34790b85baa4d52a1f3335bcf6164d2: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260806 16:00:02.398970 4489 raft_consensus.cc:348] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34790b85baa4d52a1f3335bcf6164d2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36121 } } peers { permanent_uuid: "940e4a072eb547939a7edd6fbb694754" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37769 } } peers { permanent_uuid: "4f79cbff6394473a9515069cd9a8714e" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 37777 } }
I20260806 16:00:02.399279 4489 raft_consensus.cc:374] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:00:02.399367 4489 raft_consensus.cc:729] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 940e4a072eb547939a7edd6fbb694754, State: Initialized, Role: FOLLOWER
I20260806 16:00:02.399705 4489 consensus_queue.cc:260] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34790b85baa4d52a1f3335bcf6164d2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36121 } } peers { permanent_uuid: "940e4a072eb547939a7edd6fbb694754" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37769 } } peers { permanent_uuid: "4f79cbff6394473a9515069cd9a8714e" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 37777 } }
I20260806 16:00:02.400647 4489 ts_tablet_manager.cc:1428] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 16:00:02.401010 4470 heartbeater.cc:503] Master 127.25.31.190:35621 was elected leader, sending a full tablet report...
I20260806 16:00:02.478083 4493 raft_consensus.cc:482] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:00:02.478303 4493 raft_consensus.cc:504] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34790b85baa4d52a1f3335bcf6164d2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36121 } } peers { permanent_uuid: "940e4a072eb547939a7edd6fbb694754" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37769 } } peers { permanent_uuid: "4f79cbff6394473a9515069cd9a8714e" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 37777 } }
I20260806 16:00:02.479516 4493 leader_election.cc:288] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b34790b85baa4d52a1f3335bcf6164d2 (127.25.31.129:36121), 940e4a072eb547939a7edd6fbb694754 (127.25.31.131:37769)
I20260806 16:00:02.483616 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.479785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:36121 (local address 127.25.31.130:45593)
0806 16:00:02.480008 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:02.480025 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:02.480121 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:02.480474 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:02.480478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:02.480502 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:00:02.480641 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:02.480651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.481484 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.481488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:02.482390 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:02.482400 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.483170 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.483174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:02.483290 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.483316 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 16:00:02.483392 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 16:00:02.483459 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135}
I20260806 16:00:02.483650 4482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.479986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:45593 (local address 127.25.31.129:36121)
0806 16:00:02.480105 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:02.480109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:02.480125 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:00:02.480262 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:02.480273 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:02.480330 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:02.480418 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:00:02.480772 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.481370 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.482518 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.483015 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.483146 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.483188 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:02.483428 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 16:00:02.483518 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 16:00:02.483755 4485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.480033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:42683 (local address 127.25.31.131:37769)
0806 16:00:02.480184 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:02.480189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:02.480441 (+ 252us) server_negotiation.cc:407] Connection header received
0806 16:00:02.480529 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:02.480532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:02.480596 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:02.480695 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:00:02.480987 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.481512 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.482641 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.483181 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.483295 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.483346 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:02.483561 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 16:00:02.483630 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 16:00:02.483803 4496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.479785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:37769 (local address 127.25.31.130:42683)
0806 16:00:02.480330 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:02.480344 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:02.480449 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:02.480733 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:02.480736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:02.480755 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:00:02.480875 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:02.480881 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.481633 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.481635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:02.482517 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:02.482528 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.483298 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.483301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:02.483420 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.483445 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 16:00:02.483522 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 16:00:02.483589 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":164,"thread_start_us":123,"threads_started":1}
I20260806 16:00:02.484445 4148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25a2ca471c8f48109c396ed5f2ffb532" candidate_uuid: "4f79cbff6394473a9515069cd9a8714e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b34790b85baa4d52a1f3335bcf6164d2" is_pre_election: true
I20260806 16:00:02.484447 4423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25a2ca471c8f48109c396ed5f2ffb532" candidate_uuid: "4f79cbff6394473a9515069cd9a8714e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "940e4a072eb547939a7edd6fbb694754" is_pre_election: true
I20260806 16:00:02.484755 4423 raft_consensus.cc:2454] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f79cbff6394473a9515069cd9a8714e in term 0.
I20260806 16:00:02.484755 4148 raft_consensus.cc:2454] T 25a2ca471c8f48109c396ed5f2ffb532 P b34790b85baa4d52a1f3335bcf6164d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f79cbff6394473a9515069cd9a8714e in term 0.
I20260806 16:00:02.485235 4220 leader_election.cc:302] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f79cbff6394473a9515069cd9a8714e, 940e4a072eb547939a7edd6fbb694754; no voters:
I20260806 16:00:02.485515 4493 raft_consensus.cc:2781] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:00:02.485612 4493 raft_consensus.cc:482] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:00:02.485675 4493 raft_consensus.cc:3037] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:00:02.486829 4493 raft_consensus.cc:504] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34790b85baa4d52a1f3335bcf6164d2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36121 } } peers { permanent_uuid: "940e4a072eb547939a7edd6fbb694754" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37769 } } peers { permanent_uuid: "4f79cbff6394473a9515069cd9a8714e" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 37777 } }
I20260806 16:00:02.487170 4493 leader_election.cc:288] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [CANDIDATE]: Term 1 election: Requested vote from peers b34790b85baa4d52a1f3335bcf6164d2 (127.25.31.129:36121), 940e4a072eb547939a7edd6fbb694754 (127.25.31.131:37769)
I20260806 16:00:02.487546 4148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25a2ca471c8f48109c396ed5f2ffb532" candidate_uuid: "4f79cbff6394473a9515069cd9a8714e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b34790b85baa4d52a1f3335bcf6164d2"
I20260806 16:00:02.487675 4148 raft_consensus.cc:3037] T 25a2ca471c8f48109c396ed5f2ffb532 P b34790b85baa4d52a1f3335bcf6164d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:00:02.487700 4423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25a2ca471c8f48109c396ed5f2ffb532" candidate_uuid: "4f79cbff6394473a9515069cd9a8714e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "940e4a072eb547939a7edd6fbb694754"
I20260806 16:00:02.487843 4423 raft_consensus.cc:3037] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:00:02.488855 4148 raft_consensus.cc:2454] T 25a2ca471c8f48109c396ed5f2ffb532 P b34790b85baa4d52a1f3335bcf6164d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f79cbff6394473a9515069cd9a8714e in term 1.
I20260806 16:00:02.489127 4423 raft_consensus.cc:2454] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f79cbff6394473a9515069cd9a8714e in term 1.
I20260806 16:00:02.489171 4217 leader_election.cc:302] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f79cbff6394473a9515069cd9a8714e, b34790b85baa4d52a1f3335bcf6164d2; no voters:
I20260806 16:00:02.489328 4493 raft_consensus.cc:2781] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:00:02.489562 4493 raft_consensus.cc:686] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [term 1 LEADER]: Becoming Leader. State: Replica: 4f79cbff6394473a9515069cd9a8714e, State: Running, Role: LEADER
I20260806 16:00:02.489884 4493 consensus_queue.cc:237] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34790b85baa4d52a1f3335bcf6164d2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36121 } } peers { permanent_uuid: "940e4a072eb547939a7edd6fbb694754" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37769 } } peers { permanent_uuid: "4f79cbff6394473a9515069cd9a8714e" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 37777 } }
I20260806 16:00:02.492890 3998 catalog_manager.cc:5495] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f79cbff6394473a9515069cd9a8714e (127.25.31.130). New cstate: current_term: 1 leader_uuid: "4f79cbff6394473a9515069cd9a8714e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b34790b85baa4d52a1f3335bcf6164d2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "940e4a072eb547939a7edd6fbb694754" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f79cbff6394473a9515069cd9a8714e" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 37777 } health_report { overall_health: HEALTHY } } }
I20260806 16:00:02.522740 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.519271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59708 (local address 127.25.31.130:37777)
0806 16:00:02.519411 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:02.519414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:02.519428 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:00:02.519577 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:02.519580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:02.519637 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:02.519726 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:00:02.519969 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.520466 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.521307 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:02.521539 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:02.521624 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:02.521876 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:00:02.522286 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:00:02.522447 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:02.522507 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 16:00:02.522563 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 16:00:02.528407 4148 raft_consensus.cc:1261] T 25a2ca471c8f48109c396ed5f2ffb532 P b34790b85baa4d52a1f3335bcf6164d2 [term 1 FOLLOWER]: Refusing update from remote peer 4f79cbff6394473a9515069cd9a8714e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:00:02.528407 4423 raft_consensus.cc:1261] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754 [term 1 FOLLOWER]: Refusing update from remote peer 4f79cbff6394473a9515069cd9a8714e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:00:02.529151 4497 consensus_queue.cc:1035] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b34790b85baa4d52a1f3335bcf6164d2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:00:02.529327 4493 consensus_queue.cc:1035] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [LEADER]: Connected to new peer: Peer: permanent_uuid: "940e4a072eb547939a7edd6fbb694754" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:00:02.537353 4501 mvcc.cc:205] Tried to move back new op lower bound from 7315587082349289472 to 7315587082201169920. Current Snapshot: MvccSnapshot[applied={T|T < 7315587082349289472}]
I20260806 16:00:02.537796 4500 mvcc.cc:205] Tried to move back new op lower bound from 7315587082349289472 to 7315587082201169920. Current Snapshot: MvccSnapshot[applied={T|T < 7315587082349289472}]
I20260806 16:00:02.546425 3999 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:00:02.546582 3999 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:00:02.548868 3999 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e24ebe83f824fce9e6917f1560599fc: Sending DeleteTablet for 3 replicas of tablet 25a2ca471c8f48109c396ed5f2ffb532
I20260806 16:00:02.549639 4127 tablet_service.cc:1514] Processing DeleteTablet for tablet 25a2ca471c8f48109c396ed5f2ffb532 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:00:02 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:47502
I20260806 16:00:02.549836 4264 tablet_service.cc:1514] Processing DeleteTablet for tablet 25a2ca471c8f48109c396ed5f2ffb532 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:00:02 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:59702
I20260806 16:00:02.549973 4403 tablet_service.cc:1514] Processing DeleteTablet for tablet 25a2ca471c8f48109c396ed5f2ffb532 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:00:02 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:46192
I20260806 16:00:02.550448 4511 tablet_replica.cc:330] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754: stopping tablet replica
I20260806 16:00:02.550813 4511 raft_consensus.cc:2229] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:00:02.551165 4511 raft_consensus.cc:2258] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:00:02.551272 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 4063
I20260806 16:00:02.551304 4512 tablet_replica.cc:330] T 25a2ca471c8f48109c396ed5f2ffb532 P b34790b85baa4d52a1f3335bcf6164d2: stopping tablet replica
I20260806 16:00:02.555045 4510 tablet_replica.cc:330] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e: stopping tablet replica
I20260806 16:00:02.555174 4511 ts_tablet_manager.cc:1905] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:00:02.555414 4510 raft_consensus.cc:2229] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:00:02.555855 4510 raft_consensus.cc:2258] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:00:02.557358 4510 ts_tablet_manager.cc:1905] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:00:02.557986 4511 ts_tablet_manager.cc:1918] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:00:02.558136 4511 log.cc:1196] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/25a2ca471c8f48109c396ed5f2ffb532
I20260806 16:00:02.558569 4511 ts_tablet_manager.cc:1939] T 25a2ca471c8f48109c396ed5f2ffb532 P 940e4a072eb547939a7edd6fbb694754: Deleting consensus metadata
I20260806 16:00:02.559850 3986 catalog_manager.cc:4841] TS 940e4a072eb547939a7edd6fbb694754 (127.25.31.131:37769): tablet 25a2ca471c8f48109c396ed5f2ffb532 (table test-table [id=87ea832f4e60484fb8bbc70275983796]) successfully deleted
W20260806 16:00:02.560141 3983 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:36121 (error 108)
I20260806 16:00:02.560873 4510 ts_tablet_manager.cc:1918] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:00:02.560947 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 4200
I20260806 16:00:02.561060 4510 log.cc:1196] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/25a2ca471c8f48109c396ed5f2ffb532
I20260806 16:00:02.561112 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.560822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:36121 (local address 127.0.0.1:47508)
0806 16:00:02.560949 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:02.561020 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:36121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 16:00:02.561309 3983 catalog_manager.cc:4584] TS b34790b85baa4d52a1f3335bcf6164d2 (127.25.31.129:36121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 25a2ca471c8f48109c396ed5f2ffb532: Network error: Client connection negotiation failed: client connection to 127.25.31.129:36121: connect: Connection refused (error 111)
I20260806 16:00:02.561422 4510 ts_tablet_manager.cc:1939] T 25a2ca471c8f48109c396ed5f2ffb532 P 4f79cbff6394473a9515069cd9a8714e: Deleting consensus metadata
I20260806 16:00:02.567663 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 4337
I20260806 16:00:02.569231 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:02.567791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:37777 (local address 127.0.0.1:59712)
0806 16:00:02.569065 (+ 1274us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:02.569132 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.130:37777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1161}
I20260806 16:00:02.574148 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 3967
2026-08-06T16:00:02Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2324 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:00:02 dist-test-slave-tm1g krb5kdc[4516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:00:02 dist-test-slave-tm1g krb5kdc[4516](info): set up 2 sockets
Aug 06 16:00:02 dist-test-slave-tm1g krb5kdc[4516](info): commencing operation
krb5kdc: starting...
W20260806 16:00:03.643927 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:00:03 dist-test-slave-tm1g krb5kdc[4516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:00:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:00:03Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.25.31.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.25.31.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/rangeradmin_127.25.31.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/rangeradmin_127.25.31.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.25.31.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.25.31.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/rangerlookup_127.25.31.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/rangerlookup_127.25.31.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.31.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.25.31.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/HTTP_127.25.31.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/HTTP_127.25.31.148@KRBTEST.COM.keytab.
I20260806 16:00:03.817130 25726 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/postgres -l logfile start
2026-08-06 16:00:06.339 UTC [4549] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 16:00:06.339 UTC [4549] LOG: listening on IPv4 address "127.25.31.148", port 53817
2026-08-06 16:00:06.342 UTC [4549] LOG: listening on Unix socket "/tmp/.s.PGSQL.53817"
2026-08-06 16:00:06.378 UTC [4552] LOG: database system was shut down at 2026-08-06 16:00:04 UTC
2026-08-06 16:00:06.382 UTC [4549] LOG: database system is ready to accept connections
I20260806 16:00:07.344158 25726 mini_postgres.cc:96] Postgres bound to 53817
2026-08-06 16:00:07.351 UTC [4560] FATAL: database "slave" does not exist
127.25.31.148:53817 - accepting connections
I20260806 16:00:07.352402 25726 mini_ranger.cc:161] Starting Ranger...
I20260806 16:00:07.368595 25726 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 16:00:07.829874 25726 mini_ranger.cc:87] Created ranger Postgres database
I20260806 16:00:07.829959 25726 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-admin
2026-08-06 16:00:08,125 [I] DB FLAVOR :POSTGRES
2026-08-06 16:00:08,126 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 16:00:08,126 [I] Checking connection..
2026-08-06 16:00:08,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 16:00:08,386 [I] Checking connection passed.
2026-08-06 16:00:08,386 [I] --------- Verifying version history table ---------
2026-08-06 16:00:08,386 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:00:08,628 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 16:00:08,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:00:08,881 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 16:00:08,881 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 16:00:08,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 16:00:09.123 UTC [4624] WARNING: there is no transaction in progress
2026-08-06 16:00:09,130 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 16:00:09,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:00:09,384 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 16:00:09,384 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 16:00:09,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 16:00:09,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 16:00:09,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tm1g.c.gcp-upstream.internal','N') ;"
2026-08-06 16:00:10,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 16:00:10,349 [I] Table x_portal_user does not exist in database ranger
2026-08-06 16:00:10,349 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 16:00:10,609 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 16:00:10,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 16:00:10,861 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 16:00:11,118 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 16:00:11,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 16:00:12.045 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12.060 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12.076 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12.090 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12.106 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12.122 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12.174 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12.180 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12.194 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12.203 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12.212 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12.225 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12.235 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12.241 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12.247 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12.253 UTC [4736] WARNING: there is no transaction in progress
2026-08-06 16:00:12,864 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 16:00:12,864 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 16:00:13,121 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 16:00:13,121 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 16:00:13,384 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 16:00:13,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 16:00:13,637 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 16:00:13,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 16:00:13,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 16:00:14,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tm1g.c.gcp-upstream.internal';"
2026-08-06 16:00:14,389 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 16:00:14,389 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 16:00:14,648 [I] DB_PATCHES have already been applied
I20260806 16:00:14.653980 25726 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 16:00:14.654049 25726 mini_ranger.cc:193] Using host: 127.25.31.148
I20260806 16:00:14.657855 25726 mini_ranger.cc:227] Ranger admin URL: http://127.25.31.148:60789
Aug 06, 2026 4:00:14 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:00:14 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:00:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 4:00:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 4:00:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 4:00:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 4:00:15 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 16:00:15.773488 25726 mini_ranger.cc:160] Time spent starting Ranger: real 8.421s user 0.001s sys 0.010s
Aug 06, 2026 4:00:34 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 4:00:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 4:00:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 4:00:34 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 4:00:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 4:00:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 4:00:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 4:00:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 4:00:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 4:00:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 4:00:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 4:00:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 4:00:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 4:00:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 4:00:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 4:00:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 4:00:34 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 4:00:35 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 16:00:36.296155 25726 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.25.31.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.25.31.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/rangerkms_127.25.31.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/rangerkms_127.25.31.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.31.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.25.31.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.31.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/HTTP_127.25.31.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.31.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/HTTP_127.25.31.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 16:00:36 dist-test-slave-tm1g krb5kdc[4516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032036, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 16:00:36.397222 25726 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 16:00:36.408246 25726 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 16:00:37.249886 25726 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 16:00:37.249989 25726 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-kms
I20260806 16:00:37.250089 25726 mini_ranger_kms.cc:227] Using postgres at 127.25.31.148:53817
2026-08-06 16:00:37,436 [I] DB FLAVOR :POSTGRES
2026-08-06 16:00:37,436 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 16:00:37,436 [I] Checking connection
2026-08-06 16:00:37,437 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 16:00:37,675 [I] connection success
2026-08-06 16:00:37,676 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 16:00:37,676 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 16:00:37,676 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 16:00:37,940 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 16:00:37,940 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 16:00:37,941 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 16:00:37,941 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:53817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 16:00:38,190 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 16:00:38.701239 25726 mini_ranger_kms.cc:310] Created kms service
I20260806 16:00:38.923751 25726 mini_ranger_kms.cc:326] Created kudu user
I20260806 16:00:38.971343 25726 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 16:00:38 dist-test-slave-tm1g krb5kdc[4516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032036, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:00:39.372150 25726 mini_ranger_kms.cc:384] Added ranger policy
I20260806 16:00:39.372314 25726 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 16:00:39.372337 25726 mini_ranger_kms.cc:240] Using host: 127.25.31.148
I20260806 16:00:39.376461 25726 mini_ranger_kms.cc:276] Ranger KMS PID: 4922
I20260806 16:00:39.376596 25726 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.25.31.148:45061
Aug 06, 2026 4:00:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:00:40 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:00:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 4:00:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 4:00:40 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 4:00:40 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 16:00:41.645387 25726 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.248s user 0.018s sys 0.155s
I20260806 16:00:41.645533 25726 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 16:00:41.645658 25726 mini_ranger_kms.cc:401] 127.25.31.148:45061/kms/v1/keys
Aug 06 16:00:41 dist-test-slave-tm1g krb5kdc[4516](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786032041, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.31.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 16:00:42 dist-test-slave-tm1g krb5kdc[4516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032041, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.31.148@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
Aug 06 16:00:42 dist-test-slave-tm1g krb5kdc[4516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032041, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.31.148@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
Aug 06 16:00:44 dist-test-slave-tm1g krb5kdc[4516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:00:44.977140 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:43523
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:37145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:43523
--ranger_config_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.31.148:45061/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:00:45.101404 4975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:00:45.101680 4975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:00:45.101720 4975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:00:45.105753 4975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:00:45.105806 4975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:00:45.105826 4975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:00:45.105892 4975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:00:45.105921 4975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:00:45.111236 4975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:37145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/wal
--ranger_kms_url=127.25.31.148:45061/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:43523
--ranger_config_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:43523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:00:45.112234 4975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:00:45.113240 4975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:00:45.121897 4981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:45.121934 4980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:45.121913 4983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:00:45.122113 4975 server_base.cc:1034] running on GCE node
I20260806 16:00:45.122704 4975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:00:45.123728 4975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:00:45.124888 4975 hybrid_clock.cc:648] HybridClock initialized: now 1786032045124859 us; error 41 us; skew 500 ppm
Aug 06 16:00:45 dist-test-slave-tm1g krb5kdc[4516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032045, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:00:45.132315 4975 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:00:45.133281 4975 webserver.cc:460] Webserver started at http://127.25.31.190:33839/ using document root <none> and password file <none>
I20260806 16:00:45.133839 4975 fs_manager.cc:359] Metadata directory not provided
I20260806 16:00:45.133895 4975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:00:45.134116 4975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:00:45 dist-test-slave-tm1g krb5kdc[4516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032045, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:00:45.181946 4975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "48f25dfe0eb546c7abc5caa6f2c1dcd6"
format_stamp: "Formatted at 2026-08-06 16:00:45 on dist-test-slave-tm1g"
server_key: "a644788057c1c20631130ac79ba83484"
server_key_iv: "db6092b59fcea56ef693e32603df0851"
server_key_version: "kuduclusterkey@0"
I20260806 16:00:45.182617 4975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "48f25dfe0eb546c7abc5caa6f2c1dcd6"
format_stamp: "Formatted at 2026-08-06 16:00:45 on dist-test-slave-tm1g"
server_key: "a644788057c1c20631130ac79ba83484"
server_key_iv: "db6092b59fcea56ef693e32603df0851"
server_key_version: "kuduclusterkey@0"
I20260806 16:00:45.186610 4975 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:00:45.200995 4990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:45.202514 4975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:00:45.202653 4975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "48f25dfe0eb546c7abc5caa6f2c1dcd6"
format_stamp: "Formatted at 2026-08-06 16:00:45 on dist-test-slave-tm1g"
server_key: "a644788057c1c20631130ac79ba83484"
server_key_iv: "db6092b59fcea56ef693e32603df0851"
server_key_version: "kuduclusterkey@0"
I20260806 16:00:45.202791 4975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:00:45.217705 4975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:00:45.220736 4975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:00:45.220945 4975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:00:45.229144 4975 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:43523
I20260806 16:00:45.229199 5042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:43523 every 8 connection(s)
I20260806 16:00:45.230135 4975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:00:45.233510 5043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:00:45.234849 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 4975
I20260806 16:00:45.235004 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:00:45.235327 25726 external_mini_cluster.cc:1428] Setting key 8c6e52aa7debe82c1b3920edb1821eae
I20260806 16:00:45.240569 5043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6: Bootstrap starting.
I20260806 16:00:45.243357 5043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6: Neither blocks nor log segments found. Creating new log.
Aug 06 16:00:45 dist-test-slave-tm1g krb5kdc[4516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:00:45.244300 5043 log.cc:824] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6: Log is configured to *not* fsync() on all Append() calls
I20260806 16:00:45.246906 5043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6: No bootstrap required, opened a new log
I20260806 16:00:45.249892 5046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:45.236652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34880 (local address 127.25.31.190:43523)
0806 16:00:45.237044 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:45.237065 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:45.237310 (+ 245us) server_negotiation.cc:407] Connection header received
0806 16:00:45.238078 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:45.238094 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:45.238481 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:45.238806 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:00:45.239606 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:45.240570 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:45.241450 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:45.241911 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:45.242076 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:45.244709 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:00:45.244718 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:00:45.244734 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:00:45.244766 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:00:45.247117 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:45.247547 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:45.247553 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:45.247559 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:45.247720 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:45.248008 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:45.248012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:45.248014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:45.248395 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:00:45.248596 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:45.248910 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 16:00:45.249217 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":131,"threads_started":1}
I20260806 16:00:45.250851 5043 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48f25dfe0eb546c7abc5caa6f2c1dcd6" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43523 } }
I20260806 16:00:45.251163 5043 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:00:45.251231 5043 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48f25dfe0eb546c7abc5caa6f2c1dcd6, State: Initialized, Role: FOLLOWER
I20260806 16:00:45.251693 5043 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48f25dfe0eb546c7abc5caa6f2c1dcd6" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43523 } }
I20260806 16:00:45.251848 5043 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:00:45.251910 5043 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:00:45.252002 5043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:00:45.252948 5043 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48f25dfe0eb546c7abc5caa6f2c1dcd6" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43523 } }
I20260806 16:00:45.253234 5043 leader_election.cc:302] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48f25dfe0eb546c7abc5caa6f2c1dcd6; no voters:
I20260806 16:00:45.253472 5043 leader_election.cc:288] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:00:45.253592 5048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:00:45.253839 5048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [term 1 LEADER]: Becoming Leader. State: Replica: 48f25dfe0eb546c7abc5caa6f2c1dcd6, State: Running, Role: LEADER
I20260806 16:00:45.254168 5048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48f25dfe0eb546c7abc5caa6f2c1dcd6" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43523 } }
I20260806 16:00:45.254665 5043 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:00:45.255934 5049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48f25dfe0eb546c7abc5caa6f2c1dcd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48f25dfe0eb546c7abc5caa6f2c1dcd6" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43523 } } }
I20260806 16:00:45.255911 5050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48f25dfe0eb546c7abc5caa6f2c1dcd6. Latest consensus state: current_term: 1 leader_uuid: "48f25dfe0eb546c7abc5caa6f2c1dcd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48f25dfe0eb546c7abc5caa6f2c1dcd6" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43523 } } }
I20260806 16:00:45.256063 5049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [sys.catalog]: This master's current role is: LEADER
I20260806 16:00:45.256063 5050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6 [sys.catalog]: This master's current role is: LEADER
I20260806 16:00:45.257063 5059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:00:45.257330 5043 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 16:00:45.263404 5059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:00:45.271188 5059 catalog_manager.cc:1347] Generated new cluster ID: f801d5519c994b0daad41efe320b30f1
I20260806 16:00:45.271250 5059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:00:45.277168 5059 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:00:45.278083 5059 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:00:45.292874 5059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6: Generated new TSK 0
I20260806 16:00:45.293586 5059 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 16:00:46.038883 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.31.148:45061/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:43523
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:37145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:00:46.157294 5090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:00:46.157577 5090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:00:46.157670 5090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:00:46.161911 5090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:00:46.161991 5090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:00:46.162132 5090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:00:46.168018 5090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:37145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.25.31.148:45061/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:43523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:00:46.169394 5090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:00:46.170595 5090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:00:46.181052 5099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:00:46.181139 5090 server_base.cc:1034] running on GCE node
W20260806 16:00:46.181069 5098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:46.181452 5101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:00:46.181947 5090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:00:46.182739 5090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:00:46.183972 5090 hybrid_clock.cc:648] HybridClock initialized: now 1786032046183929 us; error 48 us; skew 500 ppm
Aug 06 16:00:46 dist-test-slave-tm1g krb5kdc[4516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032046, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:00:46.187259 5090 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 16:00:46.188249 5090 webserver.cc:460] Webserver started at http://127.25.31.129:35635/ using document root <none> and password file <none>
I20260806 16:00:46.188961 5090 fs_manager.cc:359] Metadata directory not provided
I20260806 16:00:46.189071 5090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:00:46.189383 5090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:00:46 dist-test-slave-tm1g krb5kdc[4516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032046, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:00:46.208962 5090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "7186d42e12ab416eaf317683238bb475"
format_stamp: "Formatted at 2026-08-06 16:00:46 on dist-test-slave-tm1g"
server_key: "9fff5546d7e728702af1375b2f56e9a8"
server_key_iv: "c456814ae5230b4a8bd1ebc58c1938f2"
server_key_version: "kuduclusterkey@0"
I20260806 16:00:46.210556 5090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "7186d42e12ab416eaf317683238bb475"
format_stamp: "Formatted at 2026-08-06 16:00:46 on dist-test-slave-tm1g"
server_key: "9fff5546d7e728702af1375b2f56e9a8"
server_key_iv: "c456814ae5230b4a8bd1ebc58c1938f2"
server_key_version: "kuduclusterkey@0"
I20260806 16:00:46.218387 5090 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.001s
I20260806 16:00:46.250746 5108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:46.259624 5090 fs_manager.cc:725] Time spent opening block manager: real 0.023s user 0.003s sys 0.000s
I20260806 16:00:46.259799 5090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "7186d42e12ab416eaf317683238bb475"
format_stamp: "Formatted at 2026-08-06 16:00:46 on dist-test-slave-tm1g"
server_key: "9fff5546d7e728702af1375b2f56e9a8"
server_key_iv: "c456814ae5230b4a8bd1ebc58c1938f2"
server_key_version: "kuduclusterkey@0"
I20260806 16:00:46.259950 5090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:00:46.291754 5090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:00:46.295665 5090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:00:46.295979 5090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:00:46.296777 5090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:00:46.298028 5090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:00:46.298149 5090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:46.298277 5090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:00:46.298390 5090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:46.310025 5090 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:36375
I20260806 16:00:46.314000 5090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 16:00:46.318583 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 5090
I20260806 16:00:46.318711 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 16:00:46.319031 25726 external_mini_cluster.cc:1428] Setting key b5d57f6cfdcd025a00db1d71057cc382
I20260806 16:00:46.316169 5226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:36375 every 8 connection(s)
Aug 06 16:00:46 dist-test-slave-tm1g krb5kdc[4516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032046, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:00:46.337179 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:46.319970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:56549 (local address 127.25.31.190:43523)
0806 16:00:46.321932 (+ 1962us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:46.321937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:46.323310 (+ 1373us) server_negotiation.cc:407] Connection header received
0806 16:00:46.323992 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:46.323995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:46.324048 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:46.324136 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:00:46.325718 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:46.326610 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:46.328331 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:46.328720 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:46.328809 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:46.332101 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:00:46.332107 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:00:46.332113 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:00:46.332147 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:00:46.334281 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:46.335035 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:46.335042 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:46.335045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:46.335150 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:46.335646 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:46.335652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:46.335656 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:46.335931 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:00:46.336010 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:46.336792 (+ 782us) server_negotiation.cc:299] Negotiation successful
0806 16:00:46.336949 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1877,"thread_start_us":93,"threads_started":1}
I20260806 16:00:46.339214 5229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:46.318489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:43523 (local address 127.25.31.129:56549)
0806 16:00:46.322209 (+ 3720us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:46.322264 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:46.323516 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:46.324433 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:46.324443 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:46.324822 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:00:46.325541 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:46.325563 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:46.326760 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:46.326764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:46.328192 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:46.328204 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:46.329022 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:46.329026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:46.329229 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:46.329940 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:00:46.329962 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:00:46.331897 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:00:46.334458 (+ 2561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:00:46.334468 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:00:46.334499 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:00:46.334896 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:00:46.335284 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:00:46.335290 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:00:46.335294 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:00:46.335514 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:00:46.336104 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:00:46.336113 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:00:46.336472 (+ 359us) client_negotiation.cc:769] Sending connection context
0806 16:00:46.337709 (+ 1237us) client_negotiation.cc:240] Negotiation successful
0806 16:00:46.338105 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3492,"thread_start_us":163,"threads_started":1}
I20260806 16:00:46.340893 5227 heartbeater.cc:347] Connected to a master server at 127.25.31.190:43523
I20260806 16:00:46.341138 5227 heartbeater.cc:464] Registering TS with master...
I20260806 16:00:46.341709 5227 heartbeater.cc:511] Master 127.25.31.190:43523 requested a full tablet report, sending...
I20260806 16:00:46.343521 5007 ts_manager.cc:194] Registered new tserver with Master: 7186d42e12ab416eaf317683238bb475 (127.25.31.129:36375)
I20260806 16:00:46.345073 5007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:56549
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 16:00:46.374914 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.31.148:45061/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:43523
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:37145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:00:46 dist-test-slave-tm1g krb5kdc[4516](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786032046, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 16:00:46.532676 5233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:00:46.533109 5233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:00:46.533198 5233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:00:46.539186 5233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:00:46.539258 5233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:00:46.539347 5233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:00:46.545372 5233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:37145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.25.31.148:45061/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:43523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:00:46.546538 5233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:00:46.547834 5233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:00:46.559159 5242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:46.559088 5240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:46.559211 5239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:00:46.560132 5233 server_base.cc:1034] running on GCE node
I20260806 16:00:46.560683 5233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:00:46.561465 5233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:00:46.562657 5233 hybrid_clock.cc:648] HybridClock initialized: now 1786032046562586 us; error 88 us; skew 500 ppm
Aug 06 16:00:46 dist-test-slave-tm1g krb5kdc[4516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032046, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:00:46.566327 5233 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 16:00:46.567389 5233 webserver.cc:460] Webserver started at http://127.25.31.130:40007/ using document root <none> and password file <none>
I20260806 16:00:46.568123 5233 fs_manager.cc:359] Metadata directory not provided
I20260806 16:00:46.568220 5233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:00:46.568467 5233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:00:46 dist-test-slave-tm1g krb5kdc[4516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032046, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:00:46.591892 5233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "060fa5ec031142bc898ddb942abf8468"
format_stamp: "Formatted at 2026-08-06 16:00:46 on dist-test-slave-tm1g"
server_key: "6d434444e7b5d8ade72ed6575c5d3fa7"
server_key_iv: "3aead4ba4eda982f0e076131f2b59070"
server_key_version: "kuduclusterkey@0"
I20260806 16:00:46.592618 5233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "060fa5ec031142bc898ddb942abf8468"
format_stamp: "Formatted at 2026-08-06 16:00:46 on dist-test-slave-tm1g"
server_key: "6d434444e7b5d8ade72ed6575c5d3fa7"
server_key_iv: "3aead4ba4eda982f0e076131f2b59070"
server_key_version: "kuduclusterkey@0"
I20260806 16:00:46.597465 5233 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 16:00:46.619391 5249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:46.620986 5233 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260806 16:00:46.621127 5233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "060fa5ec031142bc898ddb942abf8468"
format_stamp: "Formatted at 2026-08-06 16:00:46 on dist-test-slave-tm1g"
server_key: "6d434444e7b5d8ade72ed6575c5d3fa7"
server_key_iv: "3aead4ba4eda982f0e076131f2b59070"
server_key_version: "kuduclusterkey@0"
I20260806 16:00:46.621268 5233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:00:46.651438 5233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:00:46.664602 5233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:00:46.664808 5233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:00:46.665529 5233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:00:46.666569 5233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:00:46.666626 5233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:46.666683 5233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:00:46.666711 5233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:46.676844 5233 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:38371
I20260806 16:00:46.677000 5366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:38371 every 8 connection(s)
I20260806 16:00:46.677919 5233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:00:46.686426 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 5233
I20260806 16:00:46.686563 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 16:00:46.686900 25726 external_mini_cluster.cc:1428] Setting key 47696e6ecd9ff287cd04fc7d7677158d
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
Aug 06 16:00:46 dist-test-slave-tm1g krb5kdc[4516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032046, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:00:46.720165 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:46.693672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:33481 (local address 127.25.31.190:43523)
0806 16:00:46.693803 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:46.693808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:46.706479 (+ 12671us) server_negotiation.cc:407] Connection header received
0806 16:00:46.707925 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:46.707929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:46.707986 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:46.708040 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:00:46.709793 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:46.710653 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:46.712336 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:46.712674 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:46.712747 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:46.715734 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:00:46.715738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:00:46.715740 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:00:46.715799 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:00:46.717582 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:46.718226 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:46.718230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:46.718232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:46.718359 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:46.718793 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:46.718796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:46.718798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:46.719092 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:00:46.719191 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:46.719832 (+ 641us) server_negotiation.cc:299] Negotiation successful
0806 16:00:46.719987 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 16:00:46.721442 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:46.692554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:43523 (local address 127.25.31.130:33481)
0806 16:00:46.706226 (+ 13672us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:46.706349 (+ 123us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:46.707633 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:46.708478 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:46.708488 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:46.708873 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:00:46.709607 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:46.709628 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:46.710828 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:46.710832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:46.712183 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:46.712194 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:46.712966 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:46.712969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:46.713160 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:46.713855 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:00:46.713876 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:00:46.715537 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:00:46.717720 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:00:46.717729 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:00:46.717750 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:00:46.718113 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:00:46.718485 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:00:46.718489 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:00:46.718490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:00:46.718679 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:00:46.719206 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:00:46.719213 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:00:46.719552 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 16:00:46.719775 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 16:00:46.720134 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13354,"thread_start_us":141,"threads_started":1}
I20260806 16:00:46.722730 5367 heartbeater.cc:347] Connected to a master server at 127.25.31.190:43523
I20260806 16:00:46.722985 5367 heartbeater.cc:464] Registering TS with master...
I20260806 16:00:46.723531 5367 heartbeater.cc:511] Master 127.25.31.190:43523 requested a full tablet report, sending...
I20260806 16:00:46.724938 5007 ts_manager.cc:194] Registered new tserver with Master: 060fa5ec031142bc898ddb942abf8468 (127.25.31.130:38371)
I20260806 16:00:46.725780 5007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:33481
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 16:00:46.730623 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.31.148:45061/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:43523
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:37145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:00:46.889444 5372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:00:46.889827 5372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:00:46.889940 5372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Aug 06 16:00:46 dist-test-slave-tm1g krb5kdc[4516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032046, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
W20260806 16:00:46.895630 5372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:00:46.895754 5372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:00:46.895876 5372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:00:46.903561 5372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:37145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.25.31.148:45061/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:43523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:00:46.905062 5372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:00:46.906581 5372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:00:46.919107 5378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:46.919107 5377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:00:46.919896 5380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:00:46.920320 5372 server_base.cc:1034] running on GCE node
I20260806 16:00:46.920758 5372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:00:46.921492 5372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:00:46.924801 5372 hybrid_clock.cc:648] HybridClock initialized: now 1786032046924755 us; error 44 us; skew 500 ppm
Aug 06 16:00:46 dist-test-slave-tm1g krb5kdc[4516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032046, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:00:46.928717 5372 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 16:00:46.929919 5372 webserver.cc:460] Webserver started at http://127.25.31.131:37573/ using document root <none> and password file <none>
I20260806 16:00:46.930766 5372 fs_manager.cc:359] Metadata directory not provided
I20260806 16:00:46.930934 5372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:00:46.931254 5372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:00:46 dist-test-slave-tm1g krb5kdc[4516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032046, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:00:46.955096 5372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "cabe1e85b91f46f494c6301021acf451"
format_stamp: "Formatted at 2026-08-06 16:00:46 on dist-test-slave-tm1g"
server_key: "5a3fbaf304d86e82da8412cbc0a6724e"
server_key_iv: "7ce965639770c0972640d8566fd2fc76"
server_key_version: "kuduclusterkey@0"
I20260806 16:00:46.956000 5372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "cabe1e85b91f46f494c6301021acf451"
format_stamp: "Formatted at 2026-08-06 16:00:46 on dist-test-slave-tm1g"
server_key: "5a3fbaf304d86e82da8412cbc0a6724e"
server_key_iv: "7ce965639770c0972640d8566fd2fc76"
server_key_version: "kuduclusterkey@0"
I20260806 16:00:46.961242 5372 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 16:00:46.979902 5387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:46.981520 5372 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:00:46.981653 5372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "cabe1e85b91f46f494c6301021acf451"
format_stamp: "Formatted at 2026-08-06 16:00:46 on dist-test-slave-tm1g"
server_key: "5a3fbaf304d86e82da8412cbc0a6724e"
server_key_iv: "7ce965639770c0972640d8566fd2fc76"
server_key_version: "kuduclusterkey@0"
I20260806 16:00:46.981792 5372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:00:47.011148 5372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:00:47.014891 5372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:00:47.015214 5372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:00:47.016078 5372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:00:47.017326 5372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:00:47.017401 5372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:47.017462 5372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:00:47.017496 5372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:00:47.037665 5372 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:45721
I20260806 16:00:47.037699 5503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:45721 every 8 connection(s)
I20260806 16:00:47.038705 5372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:00:47.040745 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 5372
I20260806 16:00:47.040870 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 16:00:47.041159 25726 external_mini_cluster.cc:1428] Setting key 701590d92ef244a8f0ae38e1ea8c5864
Aug 06 16:00:47 dist-test-slave-tm1g krb5kdc[4516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032046, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:00:47.053325 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.041756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:36273 (local address 127.25.31.190:43523)
0806 16:00:47.041913 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:47.041918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:47.042891 (+ 973us) server_negotiation.cc:407] Connection header received
0806 16:00:47.044022 (+ 1131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:47.044025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:47.044080 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:47.044156 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:00:47.045222 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.045778 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.047003 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.047235 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.047346 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:47.049742 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:00:47.049746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:00:47.049748 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:00:47.049770 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:00:47.051298 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:47.051808 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:47.051812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:47.051814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:47.051895 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:47.052236 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:47.052241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:47.052243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:47.052421 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:00:47.052534 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:47.053020 (+ 486us) server_negotiation.cc:299] Negotiation successful
0806 16:00:47.053148 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 16:00:47.055322 5506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.041222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:43523 (local address 127.25.31.131:36273)
0806 16:00:47.042750 (+ 1528us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:47.042805 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:47.043814 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:47.044324 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:47.044331 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:47.044608 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:00:47.045081 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:47.045094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.045906 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.045909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:47.046872 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:47.046882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.047385 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.047389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:47.047562 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:47.048212 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:00:47.048229 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:00:47.049575 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:00:47.051422 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:00:47.051428 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:00:47.051442 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:00:47.051702 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:00:47.051994 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:00:47.051997 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:00:47.051999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:00:47.052140 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:00:47.052542 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:00:47.052551 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:00:47.052787 (+ 236us) client_negotiation.cc:769] Sending connection context
0806 16:00:47.052984 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 16:00:47.053277 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1143,"thread_start_us":156,"threads_started":1}
I20260806 16:00:47.056398 5504 heartbeater.cc:347] Connected to a master server at 127.25.31.190:43523
I20260806 16:00:47.056624 5504 heartbeater.cc:464] Registering TS with master...
I20260806 16:00:47.057168 5504 heartbeater.cc:511] Master 127.25.31.190:43523 requested a full tablet report, sending...
I20260806 16:00:47.058138 5007 ts_manager.cc:194] Registered new tserver with Master: cabe1e85b91f46f494c6301021acf451 (127.25.31.131:45721)
I20260806 16:00:47.058744 5007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:36273
I20260806 16:00:47.066941 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:00:47.076851 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.069142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34888 (local address 127.25.31.190:43523)
0806 16:00:47.069267 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:47.069273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:47.069518 (+ 245us) server_negotiation.cc:407] Connection header received
0806 16:00:47.069648 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:47.069651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:47.069693 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:47.069783 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:00:47.070522 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.071051 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.071875 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.072109 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.072216 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:47.073680 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:00:47.073684 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:00:47.073687 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:00:47.073711 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:00:47.075213 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:47.075628 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:47.075633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:47.075635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:47.075720 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:00:47.075996 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:00:47.075999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:00:47.076001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:00:47.076175 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:00:47.076311 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:47.076543 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 16:00:47.076653 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 16:00:47.079751 5007 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:00:47.082984 5007 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:00:47.100265 5518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.091782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:36375 (local address 127.0.0.1:46176)
0806 16:00:47.094059 (+ 2277us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:47.094095 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:47.094191 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:47.095154 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:47.095159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:47.095185 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:00:47.095333 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:47.095340 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.096948 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.096952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:47.098424 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:47.098431 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.099658 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.099661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:47.099810 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:47.099873 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 16:00:47.099973 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 16:00:47.100103 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2129,"mutex_wait_us":25,"thread_start_us":104,"threads_started":1}
I20260806 16:00:47.101157 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.092912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46176 (local address 127.25.31.129:36375)
0806 16:00:47.094489 (+ 1577us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:47.094496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:47.094525 (+ 29us) server_negotiation.cc:407] Connection header received
0806 16:00:47.094604 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:47.094611 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:47.094821 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:47.094974 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:00:47.095476 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.096816 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.098549 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.099354 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.099451 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:47.099591 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:47.100909 (+ 1318us) server_negotiation.cc:299] Negotiation successful
0806 16:00:47.101015 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1471,"thread_start_us":122,"threads_started":1}
I20260806 16:00:47.104607 5159 tablet_service.cc:1467] Processing CreateTablet for tablet e12ba198e9c64fcbb44697253f2c58d4 (DEFAULT_TABLE table=test-table [id=a0a31630c7484dc9a4c922881482629c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:00:47.105824 5159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e12ba198e9c64fcbb44697253f2c58d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:00:47.108229 5517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.092138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:38371 (local address 127.0.0.1:54948)
0806 16:00:47.094489 (+ 2351us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:47.094502 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:47.094592 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:47.098173 (+ 3581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:47.098177 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:47.098192 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:00:47.098268 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:47.098273 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.101905 (+ 3632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.101908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:47.103009 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:47.103019 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.107818 (+ 4799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.107821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:47.107923 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:47.107947 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:00:47.108011 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 16:00:47.108087 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2298,"thread_start_us":91,"threads_started":1}
I20260806 16:00:47.109477 5519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.092705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54948 (local address 127.25.31.130:38371)
0806 16:00:47.093566 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:47.093578 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:47.094677 (+ 1099us) server_negotiation.cc:407] Connection header received
0806 16:00:47.094748 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:47.094752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:47.094902 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:47.095021 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:00:47.098433 (+ 3412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.099547 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.103170 (+ 3623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.105800 (+ 2630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.105934 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:47.106091 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:47.109211 (+ 3120us) server_negotiation.cc:299] Negotiation successful
0806 16:00:47.109322 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":127,"threads_started":1}
I20260806 16:00:47.112447 5299 tablet_service.cc:1467] Processing CreateTablet for tablet e12ba198e9c64fcbb44697253f2c58d4 (DEFAULT_TABLE table=test-table [id=a0a31630c7484dc9a4c922881482629c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:00:47.113538 5299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e12ba198e9c64fcbb44697253f2c58d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:00:47.116999 5522 tablet_bootstrap.cc:492] T e12ba198e9c64fcbb44697253f2c58d4 P 7186d42e12ab416eaf317683238bb475: Bootstrap starting.
I20260806 16:00:47.119498 5522 tablet_bootstrap.cc:654] T e12ba198e9c64fcbb44697253f2c58d4 P 7186d42e12ab416eaf317683238bb475: Neither blocks nor log segments found. Creating new log.
I20260806 16:00:47.120397 5522 log.cc:824] T e12ba198e9c64fcbb44697253f2c58d4 P 7186d42e12ab416eaf317683238bb475: Log is configured to *not* fsync() on all Append() calls
I20260806 16:00:47.122278 5516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.092511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:45721 (local address 127.0.0.1:60968)
0806 16:00:47.097468 (+ 4957us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:47.097481 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:47.097565 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:47.099661 (+ 2096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:47.099664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:47.099682 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:00:47.099786 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:47.099793 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.104526 (+ 4733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.104529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:47.106412 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:47.106423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.121784 (+ 15361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.121788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:47.121925 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:47.121956 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 16:00:47.122028 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 16:00:47.122122 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4900,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260806 16:00:47.122515 5522 tablet_bootstrap.cc:492] T e12ba198e9c64fcbb44697253f2c58d4 P 7186d42e12ab416eaf317683238bb475: No bootstrap required, opened a new log
I20260806 16:00:47.122725 5522 ts_tablet_manager.cc:1397] T e12ba198e9c64fcbb44697253f2c58d4 P 7186d42e12ab416eaf317683238bb475: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 16:00:47.122875 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.093077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60968 (local address 127.25.31.131:45721)
0806 16:00:47.093753 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:47.093766 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:47.097657 (+ 3891us) server_negotiation.cc:407] Connection header received
0806 16:00:47.097733 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:47.097738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:47.097913 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:47.098041 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:00:47.100680 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.101777 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.106617 (+ 4840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.107516 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.107602 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:47.107752 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:47.122567 (+ 14815us) server_negotiation.cc:299] Negotiation successful
0806 16:00:47.122706 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":115,"threads_started":1}
I20260806 16:00:47.126187 5437 tablet_service.cc:1467] Processing CreateTablet for tablet e12ba198e9c64fcbb44697253f2c58d4 (DEFAULT_TABLE table=test-table [id=a0a31630c7484dc9a4c922881482629c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:00:47.127389 5437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e12ba198e9c64fcbb44697253f2c58d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:00:47.127331 5522 raft_consensus.cc:348] T e12ba198e9c64fcbb44697253f2c58d4 P 7186d42e12ab416eaf317683238bb475 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7186d42e12ab416eaf317683238bb475" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36375 } } peers { permanent_uuid: "cabe1e85b91f46f494c6301021acf451" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 45721 } } peers { permanent_uuid: "060fa5ec031142bc898ddb942abf8468" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 38371 } }
I20260806 16:00:47.127619 5522 raft_consensus.cc:374] T e12ba198e9c64fcbb44697253f2c58d4 P 7186d42e12ab416eaf317683238bb475 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:00:47.127681 5522 raft_consensus.cc:729] T e12ba198e9c64fcbb44697253f2c58d4 P 7186d42e12ab416eaf317683238bb475 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7186d42e12ab416eaf317683238bb475, State: Initialized, Role: FOLLOWER
I20260806 16:00:47.128185 5522 consensus_queue.cc:260] T e12ba198e9c64fcbb44697253f2c58d4 P 7186d42e12ab416eaf317683238bb475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7186d42e12ab416eaf317683238bb475" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36375 } } peers { permanent_uuid: "cabe1e85b91f46f494c6301021acf451" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 45721 } } peers { permanent_uuid: "060fa5ec031142bc898ddb942abf8468" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 38371 } }
I20260806 16:00:47.129251 5522 ts_tablet_manager.cc:1428] T e12ba198e9c64fcbb44697253f2c58d4 P 7186d42e12ab416eaf317683238bb475: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 16:00:47.129616 5227 heartbeater.cc:503] Master 127.25.31.190:43523 was elected leader, sending a full tablet report...
I20260806 16:00:47.136921 5525 tablet_bootstrap.cc:492] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468: Bootstrap starting.
I20260806 16:00:47.138689 5526 tablet_bootstrap.cc:492] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451: Bootstrap starting.
I20260806 16:00:47.139915 5525 tablet_bootstrap.cc:654] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468: Neither blocks nor log segments found. Creating new log.
I20260806 16:00:47.140875 5525 log.cc:824] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468: Log is configured to *not* fsync() on all Append() calls
I20260806 16:00:47.142508 5526 tablet_bootstrap.cc:654] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451: Neither blocks nor log segments found. Creating new log.
I20260806 16:00:47.143121 5525 tablet_bootstrap.cc:492] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468: No bootstrap required, opened a new log
I20260806 16:00:47.143326 5525 ts_tablet_manager.cc:1397] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:00:47.143533 5526 log.cc:824] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451: Log is configured to *not* fsync() on all Append() calls
I20260806 16:00:47.147336 5525 raft_consensus.cc:348] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7186d42e12ab416eaf317683238bb475" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36375 } } peers { permanent_uuid: "cabe1e85b91f46f494c6301021acf451" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 45721 } } peers { permanent_uuid: "060fa5ec031142bc898ddb942abf8468" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 38371 } }
I20260806 16:00:47.147612 5525 raft_consensus.cc:374] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:00:47.147670 5525 raft_consensus.cc:729] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 060fa5ec031142bc898ddb942abf8468, State: Initialized, Role: FOLLOWER
I20260806 16:00:47.148183 5525 consensus_queue.cc:260] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7186d42e12ab416eaf317683238bb475" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36375 } } peers { permanent_uuid: "cabe1e85b91f46f494c6301021acf451" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 45721 } } peers { permanent_uuid: "060fa5ec031142bc898ddb942abf8468" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 38371 } }
I20260806 16:00:47.148288 5526 tablet_bootstrap.cc:492] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451: No bootstrap required, opened a new log
I20260806 16:00:47.148468 5526 ts_tablet_manager.cc:1397] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.003s
I20260806 16:00:47.149170 5525 ts_tablet_manager.cc:1428] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 16:00:47.149504 5367 heartbeater.cc:503] Master 127.25.31.190:43523 was elected leader, sending a full tablet report...
I20260806 16:00:47.155726 5526 raft_consensus.cc:348] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7186d42e12ab416eaf317683238bb475" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36375 } } peers { permanent_uuid: "cabe1e85b91f46f494c6301021acf451" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 45721 } } peers { permanent_uuid: "060fa5ec031142bc898ddb942abf8468" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 38371 } }
I20260806 16:00:47.156054 5526 raft_consensus.cc:374] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:00:47.156155 5526 raft_consensus.cc:729] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cabe1e85b91f46f494c6301021acf451, State: Initialized, Role: FOLLOWER
I20260806 16:00:47.156605 5526 consensus_queue.cc:260] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7186d42e12ab416eaf317683238bb475" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36375 } } peers { permanent_uuid: "cabe1e85b91f46f494c6301021acf451" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 45721 } } peers { permanent_uuid: "060fa5ec031142bc898ddb942abf8468" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 38371 } }
I20260806 16:00:47.157640 5526 ts_tablet_manager.cc:1428] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451: Time spent starting tablet: real 0.009s user 0.002s sys 0.004s
I20260806 16:00:47.157977 5504 heartbeater.cc:503] Master 127.25.31.190:43523 was elected leader, sending a full tablet report...
W20260806 16:00:47.181324 5368 tablet.cc:2369] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:00:47.190213 5530 raft_consensus.cc:482] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:00:47.190454 5530 raft_consensus.cc:504] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7186d42e12ab416eaf317683238bb475" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36375 } } peers { permanent_uuid: "cabe1e85b91f46f494c6301021acf451" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 45721 } } peers { permanent_uuid: "060fa5ec031142bc898ddb942abf8468" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 38371 } }
I20260806 16:00:47.197837 5530 leader_election.cc:288] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7186d42e12ab416eaf317683238bb475 (127.25.31.129:36375), 060fa5ec031142bc898ddb942abf8468 (127.25.31.130:38371)
I20260806 16:00:47.200997 5506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.195928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:36375 (local address 127.25.31.131:51023)
0806 16:00:47.196097 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:47.196114 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:47.196230 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:47.196834 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:47.196838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:47.196860 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:00:47.196987 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:47.196993 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.198019 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.198025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:47.199492 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:47.199506 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.200572 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.200576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:47.200683 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:47.200706 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 16:00:47.200778 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 16:00:47.200851 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 16:00:47.201510 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.196270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:51023 (local address 127.25.31.129:36375)
0806 16:00:47.196429 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:47.196434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:47.196449 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:00:47.196518 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:47.196522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:47.196576 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:47.196653 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:00:47.197118 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.197777 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.199692 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.200320 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.200440 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:47.200507 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:47.201281 (+ 774us) server_negotiation.cc:299] Negotiation successful
0806 16:00:47.201378 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 16:00:47.202129 5164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e12ba198e9c64fcbb44697253f2c58d4" candidate_uuid: "cabe1e85b91f46f494c6301021acf451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7186d42e12ab416eaf317683238bb475" is_pre_election: true
I20260806 16:00:47.202489 5164 raft_consensus.cc:2454] T e12ba198e9c64fcbb44697253f2c58d4 P 7186d42e12ab416eaf317683238bb475 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cabe1e85b91f46f494c6301021acf451 in term 0.
I20260806 16:00:47.202627 5519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.197202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:43493 (local address 127.25.31.130:38371)
0806 16:00:47.197335 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:47.197340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:47.198268 (+ 928us) server_negotiation.cc:407] Connection header received
0806 16:00:47.198547 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:47.198551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:47.198607 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:47.198686 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:00:47.199101 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.199797 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.201299 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.201895 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.201989 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:47.202048 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:47.202426 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 16:00:47.202505 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 16:00:47.203074 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.196855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:38371 (local address 127.25.31.131:43493)
0806 16:00:47.198183 (+ 1328us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:47.198196 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:00:47.198361 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:00:47.198798 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:00:47.198801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:00:47.198824 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:00:47.198953 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:47.198959 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.199966 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.199969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:47.201147 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:00:47.201158 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.202144 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.202147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:00:47.202255 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:47.202277 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 16:00:47.202821 (+ 544us) client_negotiation.cc:240] Negotiation successful
0806 16:00:47.202904 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1264,"thread_start_us":151,"threads_started":1}
I20260806 16:00:47.203347 5391 leader_election.cc:302] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7186d42e12ab416eaf317683238bb475, cabe1e85b91f46f494c6301021acf451; no voters:
I20260806 16:00:47.203634 5319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e12ba198e9c64fcbb44697253f2c58d4" candidate_uuid: "cabe1e85b91f46f494c6301021acf451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060fa5ec031142bc898ddb942abf8468" is_pre_election: true
I20260806 16:00:47.203886 5530 raft_consensus.cc:2781] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:00:47.203904 5319 raft_consensus.cc:2454] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cabe1e85b91f46f494c6301021acf451 in term 0.
I20260806 16:00:47.204031 5530 raft_consensus.cc:482] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:00:47.204141 5530 raft_consensus.cc:3037] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 16:00:47 dist-test-slave-tm1g krb5kdc[4516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032046, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:00:47.206341 5530 raft_consensus.cc:504] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7186d42e12ab416eaf317683238bb475" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36375 } } peers { permanent_uuid: "cabe1e85b91f46f494c6301021acf451" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 45721 } } peers { permanent_uuid: "060fa5ec031142bc898ddb942abf8468" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 38371 } }
I20260806 16:00:47.206995 5164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e12ba198e9c64fcbb44697253f2c58d4" candidate_uuid: "cabe1e85b91f46f494c6301021acf451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7186d42e12ab416eaf317683238bb475"
I20260806 16:00:47.207111 5164 raft_consensus.cc:3037] T e12ba198e9c64fcbb44697253f2c58d4 P 7186d42e12ab416eaf317683238bb475 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:00:47.208412 5164 raft_consensus.cc:2454] T e12ba198e9c64fcbb44697253f2c58d4 P 7186d42e12ab416eaf317683238bb475 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cabe1e85b91f46f494c6301021acf451 in term 1.
I20260806 16:00:47.208750 5391 leader_election.cc:302] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7186d42e12ab416eaf317683238bb475, cabe1e85b91f46f494c6301021acf451; no voters:
I20260806 16:00:47.209388 5530 leader_election.cc:288] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [CANDIDATE]: Term 1 election: Requested vote from peers 7186d42e12ab416eaf317683238bb475 (127.25.31.129:36375), 060fa5ec031142bc898ddb942abf8468 (127.25.31.130:38371)
I20260806 16:00:47.209501 5530 raft_consensus.cc:2781] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:00:47.209466 5319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e12ba198e9c64fcbb44697253f2c58d4" candidate_uuid: "cabe1e85b91f46f494c6301021acf451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "060fa5ec031142bc898ddb942abf8468"
I20260806 16:00:47.209578 5530 raft_consensus.cc:686] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [term 1 LEADER]: Becoming Leader. State: Replica: cabe1e85b91f46f494c6301021acf451, State: Running, Role: LEADER
I20260806 16:00:47.209592 5319 raft_consensus.cc:3037] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:00:47.209936 5530 consensus_queue.cc:237] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7186d42e12ab416eaf317683238bb475" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36375 } } peers { permanent_uuid: "cabe1e85b91f46f494c6301021acf451" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 45721 } } peers { permanent_uuid: "060fa5ec031142bc898ddb942abf8468" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 38371 } }
I20260806 16:00:47.210938 5319 raft_consensus.cc:2454] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cabe1e85b91f46f494c6301021acf451 in term 1.
I20260806 16:00:47.213649 5007 catalog_manager.cc:5495] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 reported cstate change: term changed from 0 to 1, leader changed from <none> to cabe1e85b91f46f494c6301021acf451 (127.25.31.131). New cstate: current_term: 1 leader_uuid: "cabe1e85b91f46f494c6301021acf451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7186d42e12ab416eaf317683238bb475" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cabe1e85b91f46f494c6301021acf451" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 45721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "060fa5ec031142bc898ddb942abf8468" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 38371 } health_report { overall_health: UNKNOWN } } }
I20260806 16:00:47.252657 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.247473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60976 (local address 127.25.31.131:45721)
0806 16:00:47.247603 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 16:00:47.247607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:00:47.247624 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:00:47.247715 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:00:47.247718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:00:47.247789 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:00:47.247900 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:00:47.248328 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.249133 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.250448 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:00:47.250779 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:00:47.250883 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:00:47.251321 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:00:47.251874 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:00:47.251996 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 16:00:47.252444 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 16:00:47.252510 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:00:47.259923 5319 raft_consensus.cc:1261] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468 [term 1 FOLLOWER]: Refusing update from remote peer cabe1e85b91f46f494c6301021acf451: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:00:47.260007 5164 raft_consensus.cc:1261] T e12ba198e9c64fcbb44697253f2c58d4 P 7186d42e12ab416eaf317683238bb475 [term 1 FOLLOWER]: Refusing update from remote peer cabe1e85b91f46f494c6301021acf451: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:00:47.260762 5533 consensus_queue.cc:1035] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7186d42e12ab416eaf317683238bb475" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:00:47.261009 5530 consensus_queue.cc:1035] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "060fa5ec031142bc898ddb942abf8468" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 38371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:00:47.272220 5536 mvcc.cc:205] Tried to move back new op lower bound from 7315587265568563200 to 7315587265375444992. Current Snapshot: MvccSnapshot[applied={T|T < 7315587265568563200}]
I20260806 16:00:47.291850 5007 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:00:47.292068 5007 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:00:47.294936 5007 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48f25dfe0eb546c7abc5caa6f2c1dcd6: Sending DeleteTablet for 3 replicas of tablet e12ba198e9c64fcbb44697253f2c58d4
I20260806 16:00:47.295912 5299 tablet_service.cc:1514] Processing DeleteTablet for tablet e12ba198e9c64fcbb44697253f2c58d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:00:47 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:54948
I20260806 16:00:47.296085 5437 tablet_service.cc:1514] Processing DeleteTablet for tablet e12ba198e9c64fcbb44697253f2c58d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:00:47 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:60968
I20260806 16:00:47.296473 5159 tablet_service.cc:1514] Processing DeleteTablet for tablet e12ba198e9c64fcbb44697253f2c58d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:00:47 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:46176
I20260806 16:00:47.297528 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 5090
I20260806 16:00:47.301748 5546 tablet_replica.cc:330] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468: stopping tablet replica
I20260806 16:00:47.302150 5546 raft_consensus.cc:2229] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:00:47.302567 5546 raft_consensus.cc:2258] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:00:47.302851 5547 tablet_replica.cc:330] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451: stopping tablet replica
I20260806 16:00:47.303234 5547 raft_consensus.cc:2229] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:00:47.303714 5547 raft_consensus.cc:2258] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:00:47.305425 5547 ts_tablet_manager.cc:1905] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 16:00:47.308986 4994 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:36375 (error 108)
I20260806 16:00:47.309038 5546 ts_tablet_manager.cc:1905] T e12ba198e9c64fcbb44697253f2c58d4 P 060fa5ec031142bc898ddb942abf8468: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:00:47.309204 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 5233
I20260806 16:00:47.311581 5516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.311224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:36375 (local address 127.0.0.1:46180)
0806 16:00:47.311412 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:47.311488 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:36375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260806 16:00:47.311847 4994 catalog_manager.cc:4584] TS 7186d42e12ab416eaf317683238bb475 (127.25.31.129:36375): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e12ba198e9c64fcbb44697253f2c58d4: Network error: Client connection negotiation failed: client connection to 127.25.31.129:36375: connect: Connection refused (error 111)
I20260806 16:00:47.312731 5547 ts_tablet_manager.cc:1918] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:00:47.312894 5547 log.cc:1196] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/e12ba198e9c64fcbb44697253f2c58d4
I20260806 16:00:47.313357 5547 ts_tablet_manager.cc:1939] T e12ba198e9c64fcbb44697253f2c58d4 P cabe1e85b91f46f494c6301021acf451: Deleting consensus metadata
I20260806 16:00:47.314728 4994 catalog_manager.cc:4841] TS cabe1e85b91f46f494c6301021acf451 (127.25.31.131:45721): tablet e12ba198e9c64fcbb44697253f2c58d4 (table test-table [id=a0a31630c7484dc9a4c922881482629c]) successfully deleted
I20260806 16:00:47.319065 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 5372
I20260806 16:00:47.319932 5516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:00:47.319208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:38371 (local address 127.0.0.1:54952)
0806 16:00:47.319789 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0806 16:00:47.319846 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.130:38371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503}
I20260806 16:00:47.326854 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 4975
I20260806 16:00:47.335484 25726 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 16:00:47.680614 25726 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 16:00:47.680751 25726 mini_ranger.cc:66] Stopping Ranger...
I20260806 16:00:48.025684 25726 mini_ranger.cc:68] Stopped Ranger
2026-08-06 16:00:48.025 UTC [4549] LOG: received smart shutdown request
2026-08-06 16:00:48.029 UTC [4549] LOG: background worker "logical replication launcher" (PID 4558) exited with exit code 1
2026-08-06 16:00:48.030 UTC [4553] LOG: shutting down
2026-08-06 16:00:48.189 UTC [4549] LOG: database system is shut down
2026-08-06T16:00:48Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45738 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:00:48 dist-test-slave-tm1g krb5kdc[5558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:00:48 dist-test-slave-tm1g krb5kdc[5558](info): set up 2 sockets
Aug 06 16:00:48 dist-test-slave-tm1g krb5kdc[5558](info): commencing operation
krb5kdc: starting...
W20260806 16:00:49.366937 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:00:49 dist-test-slave-tm1g krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:00:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:00:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.25.31.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.25.31.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/rangeradmin_127.25.31.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/rangeradmin_127.25.31.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.25.31.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.25.31.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/rangerlookup_127.25.31.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/rangerlookup_127.25.31.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.31.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.25.31.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/HTTP_127.25.31.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/HTTP_127.25.31.148@KRBTEST.COM.keytab.
I20260806 16:00:49.529101 25726 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/postgres -l logfile start
2026-08-06 16:00:52.034 UTC [5589] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 16:00:52.035 UTC [5589] LOG: listening on IPv4 address "127.25.31.148", port 44151
2026-08-06 16:00:52.038 UTC [5589] LOG: listening on Unix socket "/tmp/.s.PGSQL.44151"
2026-08-06 16:00:52.062 UTC [5593] LOG: database system was shut down at 2026-08-06 16:00:50 UTC
2026-08-06 16:00:52.066 UTC [5589] LOG: database system is ready to accept connections
I20260806 16:00:53.083251 25726 mini_postgres.cc:96] Postgres bound to 44151
2026-08-06 16:00:53.089 UTC [5601] FATAL: database "slave" does not exist
127.25.31.148:44151 - accepting connections
I20260806 16:00:53.090411 25726 mini_ranger.cc:161] Starting Ranger...
I20260806 16:00:53.105844 25726 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 16:00:53.595896 25726 mini_ranger.cc:87] Created ranger Postgres database
I20260806 16:00:53.595988 25726 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-admin
2026-08-06 16:00:53,878 [I] DB FLAVOR :POSTGRES
2026-08-06 16:00:53,879 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 16:00:53,879 [I] Checking connection..
2026-08-06 16:00:53,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 16:00:54,135 [I] Checking connection passed.
2026-08-06 16:00:54,135 [I] --------- Verifying version history table ---------
2026-08-06 16:00:54,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:00:54,378 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 16:00:54,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:00:54,621 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 16:00:54,621 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 16:00:54,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 16:00:54.879 UTC [5666] WARNING: there is no transaction in progress
2026-08-06 16:00:54,885 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 16:00:54,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 16:00:55,141 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 16:00:55,142 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 16:00:55,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 16:00:55,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 16:00:55,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tm1g.c.gcp-upstream.internal','N') ;"
2026-08-06 16:00:55,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 16:00:56,145 [I] Table x_portal_user does not exist in database ranger
2026-08-06 16:00:56,145 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 16:00:56,393 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 16:00:56,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 16:00:56,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 16:00:56,876 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 16:00:56,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 16:00:57.842 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:57.859 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:57.875 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:57.890 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:57.906 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:57.925 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:57.988 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:57.995 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:58.011 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:58.021 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:58.032 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:58.045 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:58.055 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:58.062 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:58.069 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:58.076 UTC [5777] WARNING: there is no transaction in progress
2026-08-06 16:00:58,721 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 16:00:58,721 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 16:00:58,968 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 16:00:58,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 16:00:59,228 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 16:00:59,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 16:00:59,488 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 16:00:59,488 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 16:00:59,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 16:00:59,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tm1g.c.gcp-upstream.internal';"
2026-08-06 16:01:00,247 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 16:01:00,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 16:01:00,506 [I] DB_PATCHES have already been applied
I20260806 16:01:00.512358 25726 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 16:01:00.512451 25726 mini_ranger.cc:193] Using host: 127.25.31.148
I20260806 16:01:00.516109 25726 mini_ranger.cc:227] Ranger admin URL: http://127.25.31.148:49967
Aug 06, 2026 4:01:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:01:00 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:01:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 4:01:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 4:01:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 4:01:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 4:01:01 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 16:01:01.637264 25726 mini_ranger.cc:160] Time spent starting Ranger: real 8.547s user 0.002s sys 0.007s
Aug 06, 2026 4:01:22 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 4:01:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 4:01:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 4:01:22 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 4:01:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 4:01:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 4:01:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 4:01:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 4:01:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 4:01:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 4:01:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 4:01:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 4:01:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 4:01:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 4:01:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 4:01:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 4:01:22 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 4:01:23 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 16:01:24.352596 25726 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.25.31.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.25.31.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/rangerkms_127.25.31.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.25.31.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/rangerkms_127.25.31.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.31.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.25.31.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.31.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/HTTP_127.25.31.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.31.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/HTTP_127.25.31.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 16:01:24 dist-test-slave-tm1g krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032084, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 16:01:24.485085 25726 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 16:01:24.500317 25726 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 16:01:25.451164 25726 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 16:01:25.451283 25726 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-kms
I20260806 16:01:25.451327 25726 mini_ranger_kms.cc:227] Using postgres at 127.25.31.148:44151
2026-08-06 16:01:25,838 [I] DB FLAVOR :POSTGRES
2026-08-06 16:01:25,839 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 16:01:25,839 [I] Checking connection
2026-08-06 16:01:25,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 16:01:26,407 [I] connection success
2026-08-06 16:01:26,408 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 16:01:26,408 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 16:01:26,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 16:01:26,886 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 16:01:26,886 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 16:01:26,886 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 16:01:26,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.31.148:44151/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 16:01:27,140 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 16:01:27.605795 25726 mini_ranger_kms.cc:310] Created kms service
I20260806 16:01:27.802380 25726 mini_ranger_kms.cc:326] Created kudu user
I20260806 16:01:27.845857 25726 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 16:01:27 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032084, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:01:28.255972 25726 mini_ranger_kms.cc:384] Added ranger policy
I20260806 16:01:28.256132 25726 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJ2c_Cg/build/debug/bin/postgresql.jar:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJ2c_Cg/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 16:01:28.256155 25726 mini_ranger_kms.cc:240] Using host: 127.25.31.148
I20260806 16:01:28.260604 25726 mini_ranger_kms.cc:276] Ranger KMS PID: 5963
I20260806 16:01:28.260689 25726 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.25.31.148:52941
Aug 06, 2026 4:01:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 4:01:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 4:01:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 4:01:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 4:01:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 4:01:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 16:01:30.547611 25726 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.063s user 0.027s sys 0.251s
I20260806 16:01:30.547755 25726 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 16:01:30.547860 25726 mini_ranger_kms.cc:401] 127.25.31.148:52941/kms/v1/keys
Aug 06 16:01:30 dist-test-slave-tm1g krb5kdc[5558](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786032090, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.31.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 16:01:31 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032090, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.31.148@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
Aug 06 16:01:31 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032090, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.31.148@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
Aug 06 16:01:33 dist-test-slave-tm1g krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:01:33.727519 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:45803
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:46803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:45803
--ranger_config_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.31.148:52941/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:01:33.867481 6014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:33.867729 6014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:33.867767 6014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:33.871765 6014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:01:33.871820 6014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:33.871840 6014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:33.871856 6014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:01:33.871871 6014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:01:33.877204 6014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/wal
--ranger_kms_url=127.25.31.148:52941/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:45803
--ranger_config_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:45803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:33.878230 6014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:33.879266 6014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:33.888053 6020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:33.888072 6022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:33.888041 6019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:33.888496 6014 server_base.cc:1034] running on GCE node
I20260806 16:01:33.889145 6014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:33.890512 6014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:33.891788 6014 hybrid_clock.cc:648] HybridClock initialized: now 1786032093891751 us; error 48 us; skew 500 ppm
Aug 06 16:01:33 dist-test-slave-tm1g krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032093, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:33.898576 6014 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:01:33.899510 6014 webserver.cc:460] Webserver started at http://127.25.31.190:34007/ using document root <none> and password file <none>
I20260806 16:01:33.900059 6014 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:33.900139 6014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:33.900365 6014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:01:33 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032093, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:01:33.963057 6014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "1be1196505e74df4af0f21e3d6975e48"
format_stamp: "Formatted at 2026-08-06 16:01:33 on dist-test-slave-tm1g"
server_key: "359024c0e0b7b48902daebf680b3a9bb"
server_key_iv: "ba3759c52a506dc9d6b6059bf4dff8f9"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:33.963744 6014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "1be1196505e74df4af0f21e3d6975e48"
format_stamp: "Formatted at 2026-08-06 16:01:33 on dist-test-slave-tm1g"
server_key: "359024c0e0b7b48902daebf680b3a9bb"
server_key_iv: "ba3759c52a506dc9d6b6059bf4dff8f9"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:33.967739 6014 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:01:33.984489 6030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:33.986120 6014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:01:33.986265 6014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "1be1196505e74df4af0f21e3d6975e48"
format_stamp: "Formatted at 2026-08-06 16:01:33 on dist-test-slave-tm1g"
server_key: "359024c0e0b7b48902daebf680b3a9bb"
server_key_iv: "ba3759c52a506dc9d6b6059bf4dff8f9"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:33.986441 6014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:34.037642 6014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:34.040835 6014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:34.041054 6014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:34.049232 6014 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:45803
I20260806 16:01:34.049243 6083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:45803 every 8 connection(s)
I20260806 16:01:34.050249 6014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:01:34.053045 6084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:01:34.056553 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 6014
I20260806 16:01:34.056684 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:01:34.057019 25726 external_mini_cluster.cc:1428] Setting key 1fba0eeaca9d9ea328f0c1dcaa998391
I20260806 16:01:34.058945 6084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48: Bootstrap starting.
I20260806 16:01:34.061511 6084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48: Neither blocks nor log segments found. Creating new log.
I20260806 16:01:34.062436 6084 log.cc:824] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:34.064558 6084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48: No bootstrap required, opened a new log
Aug 06 16:01:34 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:01:34.068812 6084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be1196505e74df4af0f21e3d6975e48" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 45803 } }
I20260806 16:01:34.069134 6084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:01:34.069231 6084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1be1196505e74df4af0f21e3d6975e48, State: Initialized, Role: FOLLOWER
I20260806 16:01:34.069782 6084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be1196505e74df4af0f21e3d6975e48" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 45803 } }
I20260806 16:01:34.069941 6084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:01:34.070055 6084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:01:34.070169 6084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:01:34.071235 6084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be1196505e74df4af0f21e3d6975e48" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 45803 } }
I20260806 16:01:34.071614 6084 leader_election.cc:302] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1be1196505e74df4af0f21e3d6975e48; no voters:
I20260806 16:01:34.071954 6084 leader_election.cc:288] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:01:34.072069 6090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:01:34.072381 6090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 1 LEADER]: Becoming Leader. State: Replica: 1be1196505e74df4af0f21e3d6975e48, State: Running, Role: LEADER
I20260806 16:01:34.072716 6090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be1196505e74df4af0f21e3d6975e48" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 45803 } }
I20260806 16:01:34.073521 6084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:01:34.073940 6090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1be1196505e74df4af0f21e3d6975e48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be1196505e74df4af0f21e3d6975e48" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 45803 } } }
I20260806 16:01:34.074048 6087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:34.058531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52682 (local address 127.25.31.190:45803)
0806 16:01:34.059013 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:34.059033 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:34.059075 (+ 42us) server_negotiation.cc:407] Connection header received
0806 16:01:34.059888 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:34.059912 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:34.060341 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:34.060686 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:34.061569 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:34.063010 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:34.064210 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:34.064897 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:34.065068 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:34.068282 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:34.068288 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:34.068303 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:34.068334 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:34.070377 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:34.070878 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:34.070886 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:34.070892 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:34.071027 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:34.071324 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:34.071327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:34.071331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:34.071864 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:34.072708 (+ 844us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:34.072864 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 16:01:34.073129 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":146,"threads_started":1}
I20260806 16:01:34.075474 6090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [sys.catalog]: This master's current role is: LEADER
I20260806 16:01:34.075734 6090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1be1196505e74df4af0f21e3d6975e48. Latest consensus state: current_term: 1 leader_uuid: "1be1196505e74df4af0f21e3d6975e48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be1196505e74df4af0f21e3d6975e48" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 45803 } } }
I20260806 16:01:34.075913 6090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [sys.catalog]: This master's current role is: LEADER
I20260806 16:01:34.075793 6084 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 16:01:34.077261 6105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:01:34.082994 6105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:01:34.092077 6105 catalog_manager.cc:1347] Generated new cluster ID: ef05e4c7555c435f984a5d6c51ab408d
I20260806 16:01:34.092139 6105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:01:34.108094 6105 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:01:34.109356 6105 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:01:34.125377 6105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48: Generated new TSK 0
I20260806 16:01:34.126340 6105 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 16:01:34.851094 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.31.148:52941/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:45803
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:46803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:01:34.968628 6131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:34.968902 6131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:34.968993 6131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:34.973037 6131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:34.973116 6131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:34.973232 6131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:01:34.978629 6131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.25.31.148:52941/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:45803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:34.979723 6131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:34.980769 6131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:34.990123 6137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:34.990113 6136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:34.990377 6139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:34.991302 6131 server_base.cc:1034] running on GCE node
I20260806 16:01:34.991698 6131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:34.992431 6131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:34.993608 6131 hybrid_clock.cc:648] HybridClock initialized: now 1786032094993584 us; error 44 us; skew 500 ppm
Aug 06 16:01:34 dist-test-slave-tm1g krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032094, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:34.996304 6131 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 16:01:34.997184 6131 webserver.cc:460] Webserver started at http://127.25.31.129:36953/ using document root <none> and password file <none>
I20260806 16:01:34.997792 6131 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:34.997879 6131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:34.998119 6131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:01:34 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032094, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:01:35.016232 6131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "7380626821db4c29abb3716fcda3d769"
format_stamp: "Formatted at 2026-08-06 16:01:35 on dist-test-slave-tm1g"
server_key: "3ed730a739273cd346e34194b0124b59"
server_key_iv: "4f3a55aa9be1107a9d5216afa1186885"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:35.016984 6131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "7380626821db4c29abb3716fcda3d769"
format_stamp: "Formatted at 2026-08-06 16:01:35 on dist-test-slave-tm1g"
server_key: "3ed730a739273cd346e34194b0124b59"
server_key_iv: "4f3a55aa9be1107a9d5216afa1186885"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:35.022017 6131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 16:01:35.041013 6148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:35.042687 6131 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:01:35.042814 6131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "7380626821db4c29abb3716fcda3d769"
format_stamp: "Formatted at 2026-08-06 16:01:35 on dist-test-slave-tm1g"
server_key: "3ed730a739273cd346e34194b0124b59"
server_key_iv: "4f3a55aa9be1107a9d5216afa1186885"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:35.042958 6131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:35.065124 6131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:35.068809 6131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:35.069012 6131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:35.069739 6131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:01:35.070854 6131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:01:35.070959 6131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:35.071059 6131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:01:35.071127 6131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:35.082333 6131 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:34583
I20260806 16:01:35.083576 6131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 16:01:35.088855 6264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:34583 every 8 connection(s)
I20260806 16:01:35.090032 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 6131
I20260806 16:01:35.090155 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 16:01:35.090574 25726 external_mini_cluster.cc:1428] Setting key 14fd1a8d130d16f96cc96bbe9a386173
Aug 06 16:01:35 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032094, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
I20260806 16:01:35.110590 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:35.091115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:35709 (local address 127.25.31.190:45803)
0806 16:01:35.091468 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:35.091476 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:35.092898 (+ 1422us) server_negotiation.cc:407] Connection header received
0806 16:01:35.094169 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:35.094172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:35.094231 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:35.094336 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:35.096007 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:35.096862 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:35.098569 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:35.098890 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:35.098998 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:35.102254 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:35.102258 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:35.102261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:35.102290 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:35.105623 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:35.106499 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:35.106503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:35.106505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:35.106600 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:35.107167 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:35.107171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:35.107173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:35.107399 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:35.107506 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:35.110228 (+ 2722us) server_negotiation.cc:299] Negotiation successful
0806 16:01:35.110396 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":93,"threads_started":1}
I20260806 16:01:35.112211 6268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:35.091204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:45803 (local address 127.25.31.129:35709)
0806 16:01:35.091738 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:35.092799 (+ 1061us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:35.093924 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:35.094765 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:35.094774 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:35.095124 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:35.095834 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:35.095855 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:35.097019 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:35.097022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:35.098448 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:35.098459 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:35.099072 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:35.099077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:35.099267 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:35.100067 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:35.100092 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:35.102047 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:35.105769 (+ 3722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:35.105775 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:35.105790 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:35.106370 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:35.106720 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:35.106723 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:35.106725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:35.106893 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:35.109543 (+ 2650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:35.109552 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:35.109900 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 16:01:35.110794 (+ 894us) client_negotiation.cc:240] Negotiation successful
0806 16:01:35.111170 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":165,"threads_started":1}
I20260806 16:01:35.114892 6265 heartbeater.cc:347] Connected to a master server at 127.25.31.190:45803
I20260806 16:01:35.115172 6265 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:35.115677 6265 heartbeater.cc:511] Master 127.25.31.190:45803 requested a full tablet report, sending...
I20260806 16:01:35.117187 6047 ts_manager.cc:194] Registered new tserver with Master: 7380626821db4c29abb3716fcda3d769 (127.25.31.129:34583)
I20260806 16:01:35.118371 6047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:35709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 16:01:35.133383 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.31.148:52941/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:45803
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:46803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:01:35 dist-test-slave-tm1g krb5kdc[5558](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786032095, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 16:01:35.287917 6271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:35.288182 6271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:35.288265 6271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:35.293107 6271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:35.293190 6271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:35.293274 6271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:01:35.299935 6271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.25.31.148:52941/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:45803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:35.301082 6271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:35.302117 6271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:35.312641 6280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:35.312728 6279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:35.313917 6282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:35.314347 6271 server_base.cc:1034] running on GCE node
I20260806 16:01:35.314878 6271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:35.315629 6271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:35.316807 6271 hybrid_clock.cc:648] HybridClock initialized: now 1786032095316774 us; error 55 us; skew 500 ppm
Aug 06 16:01:35 dist-test-slave-tm1g krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032095, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:35.319813 6271 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 16:01:35.320742 6271 webserver.cc:460] Webserver started at http://127.25.31.130:36535/ using document root <none> and password file <none>
I20260806 16:01:35.321333 6271 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:35.321425 6271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:35.321681 6271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:01:35 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032095, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:01:35.336886 6271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "cf8644f287b54ee5a074ae67dad8c314"
format_stamp: "Formatted at 2026-08-06 16:01:35 on dist-test-slave-tm1g"
server_key: "3d10b806d1c402bdecf5c188635c1bd4"
server_key_iv: "03ae2858dfe98849a4746195f8a47123"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:35.337637 6271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "cf8644f287b54ee5a074ae67dad8c314"
format_stamp: "Formatted at 2026-08-06 16:01:35 on dist-test-slave-tm1g"
server_key: "3d10b806d1c402bdecf5c188635c1bd4"
server_key_iv: "03ae2858dfe98849a4746195f8a47123"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:35.344241 6271 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 16:01:35.370750 6289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:35.372659 6271 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.003s sys 0.001s
I20260806 16:01:35.372835 6271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "cf8644f287b54ee5a074ae67dad8c314"
format_stamp: "Formatted at 2026-08-06 16:01:35 on dist-test-slave-tm1g"
server_key: "3d10b806d1c402bdecf5c188635c1bd4"
server_key_iv: "03ae2858dfe98849a4746195f8a47123"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:35.373045 6271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:35.392778 6271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:35.396597 6271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:35.396863 6271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:35.397661 6271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:01:35.398784 6271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:01:35.398892 6271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:35.398994 6271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:01:35.399058 6271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:35.413036 6271 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:45599
I20260806 16:01:35.414558 6271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:01:35.424315 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 6271
I20260806 16:01:35.424453 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 16:01:35.424760 25726 external_mini_cluster.cc:1428] Setting key 173a922cfbee2897c6dfeba2497631fe
I20260806 16:01:35.416635 6405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:45599 every 8 connection(s)
Aug 06 16:01:35 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032095, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:01:35.440393 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:35.418558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:58727 (local address 127.25.31.190:45803)
0806 16:01:35.418706 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:35.418710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:35.421241 (+ 2531us) server_negotiation.cc:407] Connection header received
0806 16:01:35.422560 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:35.422565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:35.422627 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:35.422686 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:35.429269 (+ 6583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:35.430141 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:35.431959 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:35.432302 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:35.432389 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:35.435911 (+ 3522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:35.435915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:35.435918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:35.435944 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:35.437711 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:35.438402 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:35.438406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:35.438408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:35.438500 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:35.438938 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:35.438942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:35.438944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:35.439203 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:35.439293 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:35.440029 (+ 736us) server_negotiation.cc:299] Negotiation successful
0806 16:01:35.440190 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 16:01:35.442584 6408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:35.419449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:45803 (local address 127.25.31.130:58727)
0806 16:01:35.421063 (+ 1614us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:35.421121 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:35.422254 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:35.423109 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:35.423120 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:35.423508 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:35.429067 (+ 5559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:35.429093 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:35.430342 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:35.430346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:35.431811 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:35.431823 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:35.432629 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:35.432633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:35.432856 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:35.433613 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:35.433639 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:35.435694 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:35.437846 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:35.437853 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:35.437870 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:35.438236 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:35.438630 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:35.438634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:35.438636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:35.438821 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:35.439367 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:35.439375 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:35.439727 (+ 352us) client_negotiation.cc:769] Sending connection context
0806 16:01:35.441105 (+ 1378us) client_negotiation.cc:240] Negotiation successful
0806 16:01:35.441466 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1387,"thread_start_us":156,"threads_started":1}
I20260806 16:01:35.444190 6406 heartbeater.cc:347] Connected to a master server at 127.25.31.190:45803
I20260806 16:01:35.444434 6406 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:35.445019 6406 heartbeater.cc:511] Master 127.25.31.190:45803 requested a full tablet report, sending...
I20260806 16:01:35.446269 6047 ts_manager.cc:194] Registered new tserver with Master: cf8644f287b54ee5a074ae67dad8c314 (127.25.31.130:45599)
I20260806 16:01:35.447075 6047 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:58727
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 16:01:35.471522 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.31.148:52941/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:45803
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:46803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:01:35.615337 6411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:35.615823 6411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:35.615971 6411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:35.622035 6411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:35.622165 6411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:35.622323 6411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:01:35.630126 6411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.25.31.148:52941/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:45803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:35.631716 6411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:35.633117 6411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:35.647150 6416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:35.648425 6417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:35.650477 6419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:35.655350 6411 server_base.cc:1034] running on GCE node
I20260806 16:01:35.655908 6411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:35.656819 6411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:35.673708 6411 hybrid_clock.cc:648] HybridClock initialized: now 1786032095673648 us; error 79 us; skew 500 ppm
Aug 06 16:01:35 dist-test-slave-tm1g krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032095, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:35.677704 6411 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 16:01:35.678918 6411 webserver.cc:460] Webserver started at http://127.25.31.131:40451/ using document root <none> and password file <none>
I20260806 16:01:35.679659 6411 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:35.681555 6411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:35.681866 6411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 16:01:35 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032095, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:01:35.704274 6411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "52ca4f1f8ad34fc4b82104ca1111969e"
format_stamp: "Formatted at 2026-08-06 16:01:35 on dist-test-slave-tm1g"
server_key: "671c388160ada0fb2bd1a2c191c164fb"
server_key_iv: "1aa328bb560ccfdd5ee2dc1808ff54af"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:35.706452 6411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "52ca4f1f8ad34fc4b82104ca1111969e"
format_stamp: "Formatted at 2026-08-06 16:01:35 on dist-test-slave-tm1g"
server_key: "671c388160ada0fb2bd1a2c191c164fb"
server_key_iv: "1aa328bb560ccfdd5ee2dc1808ff54af"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:35.711323 6411 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 16:01:35.731995 6426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:35.733722 6411 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:01:35.733863 6411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "52ca4f1f8ad34fc4b82104ca1111969e"
format_stamp: "Formatted at 2026-08-06 16:01:35 on dist-test-slave-tm1g"
server_key: "671c388160ada0fb2bd1a2c191c164fb"
server_key_iv: "1aa328bb560ccfdd5ee2dc1808ff54af"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:35.735639 6411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:35.764086 6411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:35.767928 6411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:35.768328 6411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:35.769068 6411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:01:35.770123 6411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:01:35.770222 6411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:35.770334 6411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:01:35.770409 6411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:35.781639 6411 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:38757
I20260806 16:01:35.782846 6411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:01:35.783053 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 6411
I20260806 16:01:35.783188 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 16:01:35.783514 25726 external_mini_cluster.cc:1428] Setting key 4d3612ab4a878ad101fb88ebbbeb4ed1
I20260806 16:01:35.785720 6542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:38757 every 8 connection(s)
Aug 06 16:01:35 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032095, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
Aug 06 16:01:35 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032095, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:01:35.803552 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:35.790261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:47067 (local address 127.25.31.190:45803)
0806 16:01:35.790404 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:35.790409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:35.790605 (+ 196us) server_negotiation.cc:407] Connection header received
0806 16:01:35.791746 (+ 1141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:35.791750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:35.791804 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:35.791857 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:35.793421 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:35.794252 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:35.796091 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:35.796409 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:35.796486 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:35.799514 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:35.799518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:35.799521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:35.799548 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:35.801218 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:35.801817 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:35.801820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:35.801822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:35.801905 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:35.802282 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:35.802286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:35.802287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:35.802528 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:35.802597 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:35.803264 (+ 667us) server_negotiation.cc:299] Negotiation successful
0806 16:01:35.803392 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 16:01:35.805270 6545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:35.789237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:45803 (local address 127.25.31.131:47067)
0806 16:01:35.790470 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:35.790524 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:35.791516 (+ 992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:35.792252 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:35.792260 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:35.792614 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:35.793250 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:35.793268 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:35.794472 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:35.794475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:35.795954 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:35.795965 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:35.796692 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:35.796696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:35.796891 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:35.797593 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:35.797613 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:35.799323 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:35.801356 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:35.801362 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:35.801389 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:35.801705 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:35.802010 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:35.802013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:35.802015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:35.802181 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:35.802676 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:35.802681 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:35.802974 (+ 293us) client_negotiation.cc:769] Sending connection context
0806 16:01:35.803941 (+ 967us) client_negotiation.cc:240] Negotiation successful
0806 16:01:35.804266 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"thread_start_us":145,"threads_started":1}
I20260806 16:01:35.806797 6543 heartbeater.cc:347] Connected to a master server at 127.25.31.190:45803
I20260806 16:01:35.807026 6543 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:35.807566 6543 heartbeater.cc:511] Master 127.25.31.190:45803 requested a full tablet report, sending...
I20260806 16:01:35.808734 6046 ts_manager.cc:194] Registered new tserver with Master: 52ca4f1f8ad34fc4b82104ca1111969e (127.25.31.131:38757)
I20260806 16:01:35.809547 6046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:47067
I20260806 16:01:35.812255 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:01:35.813139 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 6131
I20260806 16:01:35.827812 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 6271
I20260806 16:01:35.836866 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 6411
I20260806 16:01:35.847678 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 6014
I20260806 16:01:35.857910 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:45803
--webserver_interface=127.25.31.190
--webserver_port=34007
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:46803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:45803
--ranger_config_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.31.148:52941/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.31.148:52941/kms with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:01:35.978816 6546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:35.979094 6546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:35.979166 6546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:35.983145 6546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:01:35.983222 6546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:35.983260 6546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:35.983316 6546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:01:35.983354 6546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:01:35.988690 6546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.31.148:52941/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:45803
--ranger_config_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:45803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=34007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:35.989670 6546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:35.990721 6546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:35.998935 6551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:35.998982 6552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:35.999028 6546 server_base.cc:1034] running on GCE node
W20260806 16:01:35.998966 6554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:35.999675 6546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:36.000661 6546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:36.001840 6546 hybrid_clock.cc:648] HybridClock initialized: now 1786032096001827 us; error 27 us; skew 500 ppm
Aug 06 16:01:35 dist-test-slave-tm1g krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032095, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:36.004578 6546 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:01:36.005438 6546 webserver.cc:460] Webserver started at http://127.25.31.190:34007/ using document root <none> and password file <none>
I20260806 16:01:36.005971 6546 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:36.006070 6546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 16:01:36 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032095, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:01:36.090348 6546 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:01:36.093585 6563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:36.094965 6546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:01:36.095082 6546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "1be1196505e74df4af0f21e3d6975e48"
format_stamp: "Formatted at 2026-08-06 16:01:33 on dist-test-slave-tm1g"
server_key: "359024c0e0b7b48902daebf680b3a9bb"
server_key_iv: "ba3759c52a506dc9d6b6059bf4dff8f9"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:36.095608 6546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:36.116832 6546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:36.119524 6546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:36.119675 6546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:36.126791 6546 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:45803
I20260806 16:01:36.126796 6615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:45803 every 8 connection(s)
I20260806 16:01:36.127839 6546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:01:36.131939 6616 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:01:36.133464 6616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48: Bootstrap starting.
I20260806 16:01:36.137943 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 6546
I20260806 16:01:36.138500 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.31.148:52941/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:34583
--local_ip_for_outbound_sockets=127.25.31.129
--tserver_master_addrs=127.25.31.190:45803
--webserver_port=36953
--webserver_interface=127.25.31.129
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:46803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.31.148:52941/kms with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:01:36.150573 6616 log.cc:824] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:36.157944 6616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:01:36.158381 6616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48: Bootstrap complete.
I20260806 16:01:36.162447 6616 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be1196505e74df4af0f21e3d6975e48" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 45803 } }
I20260806 16:01:36.162869 6616 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1be1196505e74df4af0f21e3d6975e48, State: Initialized, Role: FOLLOWER
I20260806 16:01:36.163385 6616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be1196505e74df4af0f21e3d6975e48" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 45803 } }
I20260806 16:01:36.163535 6616 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:01:36.163625 6616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:01:36.163722 6616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 1 FOLLOWER]: Advancing to term 2
I20260806 16:01:36.165547 6616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be1196505e74df4af0f21e3d6975e48" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 45803 } }
I20260806 16:01:36.165865 6616 leader_election.cc:302] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1be1196505e74df4af0f21e3d6975e48; no voters:
I20260806 16:01:36.166169 6616 leader_election.cc:288] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 16:01:36.166297 6620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 16:01:36.166538 6620 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [term 2 LEADER]: Becoming Leader. State: Replica: 1be1196505e74df4af0f21e3d6975e48, State: Running, Role: LEADER
I20260806 16:01:36.166880 6620 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be1196505e74df4af0f21e3d6975e48" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 45803 } }
I20260806 16:01:36.167332 6616 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:01:36.167790 6620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1be1196505e74df4af0f21e3d6975e48. Latest consensus state: current_term: 2 leader_uuid: "1be1196505e74df4af0f21e3d6975e48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be1196505e74df4af0f21e3d6975e48" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 45803 } } }
I20260806 16:01:36.167972 6620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [sys.catalog]: This master's current role is: LEADER
I20260806 16:01:36.168202 6621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1be1196505e74df4af0f21e3d6975e48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be1196505e74df4af0f21e3d6975e48" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 45803 } } }
I20260806 16:01:36.168386 6621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48 [sys.catalog]: This master's current role is: LEADER
I20260806 16:01:36.168372 6625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:01:36.171669 6625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:01:36.172142 6616 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 16:01:36.172329 6625 catalog_manager.cc:1259] Loaded cluster ID: ef05e4c7555c435f984a5d6c51ab408d
I20260806 16:01:36.172376 6625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:01:36.174726 6625 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:01:36.175376 6625 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48: Loaded TSK: 0
I20260806 16:01:36.176537 6625 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:01:36.276512 6618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:36.276770 6618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:36.276865 6618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:36.282007 6618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:36.282145 6618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:36.282284 6618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:01:36.288693 6618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.31.148:52941/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:34583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=36953
--tserver_master_addrs=127.25.31.190:45803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:36.290092 6618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:36.291154 6618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:36.300993 6653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:36.301168 6654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:36.301677 6618 server_base.cc:1034] running on GCE node
W20260806 16:01:36.301975 6656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:36.302635 6618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:36.303408 6618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:36.304584 6618 hybrid_clock.cc:648] HybridClock initialized: now 1786032096304557 us; error 40 us; skew 500 ppm
Aug 06 16:01:36 dist-test-slave-tm1g krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032096, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:36.307655 6618 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 16:01:36.308822 6618 webserver.cc:460] Webserver started at http://127.25.31.129:36953/ using document root <none> and password file <none>
I20260806 16:01:36.309525 6618 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:36.309640 6618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 16:01:36 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032096, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:01:36.342259 6618 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.003s sys 0.000s
I20260806 16:01:36.345847 6664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:36.347329 6618 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:01:36.347479 6618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "7380626821db4c29abb3716fcda3d769"
format_stamp: "Formatted at 2026-08-06 16:01:35 on dist-test-slave-tm1g"
server_key: "3ed730a739273cd346e34194b0124b59"
server_key_iv: "4f3a55aa9be1107a9d5216afa1186885"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:36.348065 6618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:36.383307 6618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:36.387094 6618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:36.387351 6618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:36.388100 6618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:01:36.389189 6618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:01:36.389295 6618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:36.389393 6618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:01:36.389456 6618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:36.405083 6618 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:34583
I20260806 16:01:36.406678 6618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 16:01:36.409826 6780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:34583 every 8 connection(s)
I20260806 16:01:36.416705 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 6618
I20260806 16:01:36.417248 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.31.148:52941/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:45599
--local_ip_for_outbound_sockets=127.25.31.130
--tserver_master_addrs=127.25.31.190:45803
--webserver_port=36535
--webserver_interface=127.25.31.130
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:46803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.31.148:52941/kms with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:01:36 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032096, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:01:36.431605 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:36.412213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:52109 (local address 127.25.31.190:45803)
0806 16:01:36.412760 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:36.412783 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:36.412829 (+ 46us) server_negotiation.cc:407] Connection header received
0806 16:01:36.413128 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:36.413141 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:36.413538 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:36.413877 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:36.415227 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:36.419045 (+ 3818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:36.420741 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:36.421455 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:36.421610 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:36.424685 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:36.424692 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:36.424707 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:36.424738 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:36.426864 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:36.427489 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:36.427495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:36.427501 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:36.427638 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:36.428036 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:36.428039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:36.428041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:36.428539 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:36.428667 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:36.430514 (+ 1847us) server_negotiation.cc:299] Negotiation successful
0806 16:01:36.430795 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":141,"threads_started":1}
I20260806 16:01:36.432169 6784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:36.410821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:45803 (local address 127.25.31.129:52109)
0806 16:01:36.411312 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:36.411365 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:36.412397 (+ 1032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:36.413990 (+ 1593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:36.413999 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:36.414381 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:36.415028 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:36.415049 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:36.419209 (+ 4160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:36.419212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:36.420595 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:36.420606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:36.421819 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:36.421822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:36.422007 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:36.422707 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:36.422727 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:36.424472 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:36.426999 (+ 2527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:36.427005 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:36.427019 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:36.427343 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:36.427750 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:36.427753 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:36.427755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:36.427923 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:36.428724 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:36.428731 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:36.429009 (+ 278us) client_negotiation.cc:769] Sending connection context
0806 16:01:36.429196 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 16:01:36.429525 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":139,"threads_started":1}
I20260806 16:01:36.435564 6782 heartbeater.cc:347] Connected to a master server at 127.25.31.190:45803
I20260806 16:01:36.435813 6782 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:36.436395 6782 heartbeater.cc:511] Master 127.25.31.190:45803 requested a full tablet report, sending...
W20260806 16:01:36.438591 6782 heartbeater.cc:644] Failed to heartbeat to 127.25.31.190:45803 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 16:01:36.438711 6782 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:36.438869 6782 heartbeater.cc:511] Master 127.25.31.190:45803 requested a full tablet report, sending...
I20260806 16:01:36.439685 6782 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:36.439847 6782 heartbeater.cc:511] Master 127.25.31.190:45803 requested a full tablet report, sending...
W20260806 16:01:36.440590 6782 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:01:36.581004 6786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:36.581284 6786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:36.581349 6786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:36.585579 6786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:36.585659 6786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:36.585807 6786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:01:36.592165 6786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.31.148:52941/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:45599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=36535
--tserver_master_addrs=127.25.31.190:45803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:36.593338 6786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:36.594461 6786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:36.605425 6791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:36.606241 6792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:36.606601 6794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:36.607021 6786 server_base.cc:1034] running on GCE node
I20260806 16:01:36.607537 6786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:36.608361 6786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:36.609562 6786 hybrid_clock.cc:648] HybridClock initialized: now 1786032096609527 us; error 42 us; skew 500 ppm
Aug 06 16:01:36 dist-test-slave-tm1g krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032096, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:36.613268 6786 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 16:01:36.614423 6786 webserver.cc:460] Webserver started at http://127.25.31.130:36535/ using document root <none> and password file <none>
I20260806 16:01:36.615146 6786 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:36.615311 6786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 16:01:36 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032096, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:01:36.653705 6786 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.003s sys 0.000s
I20260806 16:01:36.658123 6802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:36.659550 6786 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 16:01:36.659693 6786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "cf8644f287b54ee5a074ae67dad8c314"
format_stamp: "Formatted at 2026-08-06 16:01:35 on dist-test-slave-tm1g"
server_key: "3d10b806d1c402bdecf5c188635c1bd4"
server_key_iv: "03ae2858dfe98849a4746195f8a47123"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:36.660279 6786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:36.701112 6786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:36.704811 6786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:36.705027 6786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:36.705753 6786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:01:36.706789 6786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:01:36.706869 6786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:36.706929 6786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:01:36.706965 6786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:36.717903 6786 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:45599
I20260806 16:01:36.718008 6919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:45599 every 8 connection(s)
I20260806 16:01:36.719094 6786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:01:36.723735 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 6786
I20260806 16:01:36.724242 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.31.148:52941/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:38757
--local_ip_for_outbound_sockets=127.25.31.131
--tserver_master_addrs=127.25.31.190:45803
--webserver_port=40451
--webserver_interface=127.25.31.131
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:46803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.31.148:52941/kms with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:01:36 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032096, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:01:36.738647 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:36.721118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:36111 (local address 127.25.31.190:45803)
0806 16:01:36.722514 (+ 1396us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:36.722519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:36.722537 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:01:36.723178 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:36.723182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:36.723244 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:36.723325 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:36.724799 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:36.725665 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:36.728886 (+ 3221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:36.729268 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:36.729351 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:36.732679 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:36.732683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:36.732688 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:36.732717 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:36.736010 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:36.736657 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:36.736662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:36.736664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:36.736750 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:36.737192 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:36.737196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:36.737198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:36.737460 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:36.737555 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:36.738293 (+ 738us) server_negotiation.cc:299] Negotiation successful
0806 16:01:36.738467 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1300}
I20260806 16:01:36.740502 6922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:36.721528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:45803 (local address 127.25.31.130:36111)
0806 16:01:36.721995 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:36.722047 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:36.722973 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:36.723552 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:36.723563 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:36.723912 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:36.724598 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:36.724619 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:36.727295 (+ 2676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:36.727299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:36.728728 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:36.728739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:36.729557 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:36.729561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:36.729771 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:36.730566 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:36.730587 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:36.732469 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:36.736155 (+ 3686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:36.736161 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:36.736177 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:36.736533 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:36.736867 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:36.736870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:36.736873 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:36.737061 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:36.737629 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:36.737638 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:36.737989 (+ 351us) client_negotiation.cc:769] Sending connection context
0806 16:01:36.739106 (+ 1117us) client_negotiation.cc:240] Negotiation successful
0806 16:01:36.739423 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":140,"threads_started":1}
I20260806 16:01:36.741750 6920 heartbeater.cc:347] Connected to a master server at 127.25.31.190:45803
I20260806 16:01:36.741993 6920 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:36.742533 6920 heartbeater.cc:511] Master 127.25.31.190:45803 requested a full tablet report, sending...
W20260806 16:01:36.744089 6920 heartbeater.cc:644] Failed to heartbeat to 127.25.31.190:45803 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 16:01:36.744211 6920 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:36.744376 6920 heartbeater.cc:511] Master 127.25.31.190:45803 requested a full tablet report, sending...
I20260806 16:01:36.745199 6920 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:36.745410 6920 heartbeater.cc:511] Master 127.25.31.190:45803 requested a full tablet report, sending...
W20260806 16:01:36.746187 6920 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:01:36.861462 6923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:36.861707 6923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:36.861749 6923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:36.866012 6923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:36.866070 6923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:36.866153 6923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:01:36.872838 6923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.31.148:52941/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:38757
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=40451
--tserver_master_addrs=127.25.31.190:45803
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:36.874290 6923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:36.875670 6923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:36.887290 6929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:36.887377 6931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:36.887284 6928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:36.887979 6923 server_base.cc:1034] running on GCE node
I20260806 16:01:36.888387 6923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:36.889012 6923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:36.890208 6923 hybrid_clock.cc:648] HybridClock initialized: now 1786032096890180 us; error 43 us; skew 500 ppm
Aug 06 16:01:36 dist-test-slave-tm1g krb5kdc[5558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032096, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:36.893532 6923 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 16:01:36.894590 6923 webserver.cc:460] Webserver started at http://127.25.31.131:40451/ using document root <none> and password file <none>
I20260806 16:01:36.895151 6923 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:36.895258 6923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 16:01:36 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032096, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:01:36.919754 6923 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 16:01:36.925288 6939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:36.926915 6923 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260806 16:01:36.927120 6923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "52ca4f1f8ad34fc4b82104ca1111969e"
format_stamp: "Formatted at 2026-08-06 16:01:35 on dist-test-slave-tm1g"
server_key: "671c388160ada0fb2bd1a2c191c164fb"
server_key_iv: "1aa328bb560ccfdd5ee2dc1808ff54af"
server_key_version: "kuduclusterkey@0"
I20260806 16:01:36.927933 6923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:36.944427 6923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:36.947404 6923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:36.947606 6923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:36.948246 6923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:01:36.949074 6923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:01:36.949120 6923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:36.949159 6923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:01:36.949214 6923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:36.960083 6923 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:38757
I20260806 16:01:36.960199 7057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:38757 every 8 connection(s)
I20260806 16:01:36.961344 6923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:01:36.962906 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 6923
Aug 06 16:01:36 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032096, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:01:36.981082 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:36.965246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52686 (local address 127.25.31.190:45803)
0806 16:01:36.965719 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:36.965728 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:36.965747 (+ 19us) server_negotiation.cc:407] Connection header received
0806 16:01:36.965801 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:36.965805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:36.965855 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:36.965956 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:36.966963 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:36.967789 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:36.969083 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:36.969457 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:36.969549 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:36.971333 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:36.971337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:36.971339 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:36.971360 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:36.978882 (+ 7522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:36.979619 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:36.979623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:36.979626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:36.979716 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:36.980037 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:36.980040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:36.980044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:36.980279 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:36.980471 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:36.980723 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 16:01:36.980854 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":211,"server-negotiator.queue_time_us":401,"thread_start_us":118,"threads_started":1}
I20260806 16:01:36.981935 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:36.965245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:38693 (local address 127.25.31.190:45803)
0806 16:01:36.965390 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:36.965394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:36.966118 (+ 724us) server_negotiation.cc:407] Connection header received
0806 16:01:36.967245 (+ 1127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:36.967248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:36.967294 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:36.967377 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:36.968638 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:36.969205 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:36.970782 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:36.971017 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:36.971125 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:36.975316 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:36.975320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:36.975323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:36.975367 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:36.979670 (+ 4303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:36.980271 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:36.980274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:36.980275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:36.980351 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:36.980755 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:36.980758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:36.980760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:36.980939 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:36.981015 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:36.981696 (+ 681us) server_negotiation.cc:299] Negotiation successful
0806 16:01:36.981795 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 16:01:36.983342 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:36.964157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:45803 (local address 127.25.31.131:38693)
0806 16:01:36.965990 (+ 1833us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:36.966045 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:36.967001 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:36.967537 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:36.967545 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:36.967857 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:36.968499 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:36.968515 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:36.969338 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:36.969341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:36.970658 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:36.970667 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:36.971164 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:36.971169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:36.971362 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:36.973309 (+ 1947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:36.973329 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:36.975128 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:36.979804 (+ 4676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:36.979810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:36.979825 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:36.980158 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:36.980455 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:36.980467 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:36.980469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:36.980645 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:36.981083 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:36.981090 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:36.981404 (+ 314us) client_negotiation.cc:769] Sending connection context
0806 16:01:36.982119 (+ 715us) client_negotiation.cc:240] Negotiation successful
0806 16:01:36.982441 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1613,"thread_start_us":138,"threads_started":1}
I20260806 16:01:36.984409 7058 heartbeater.cc:347] Connected to a master server at 127.25.31.190:45803
I20260806 16:01:36.984685 7058 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:36.985278 7058 heartbeater.cc:511] Master 127.25.31.190:45803 requested a full tablet report, sending...
I20260806 16:01:36.986702 6580 ts_manager.cc:194] Registered new tserver with Master: 52ca4f1f8ad34fc4b82104ca1111969e (127.25.31.131:38757)
I20260806 16:01:36.987918 6580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:38693
Aug 06 16:01:37 dist-test-slave-tm1g krb5kdc[5558](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786032097, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:37.440852 6782 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:37.441169 6782 heartbeater.cc:511] Master 127.25.31.190:45803 requested a full tablet report, sending...
I20260806 16:01:37.442171 6579 ts_manager.cc:194] Registered new tserver with Master: 7380626821db4c29abb3716fcda3d769 (127.25.31.129:34583)
I20260806 16:01:37.443117 6579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:52109
Aug 06 16:01:37 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032097, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:01:37.746425 6920 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:37.746743 6920 heartbeater.cc:511] Master 127.25.31.190:45803 requested a full tablet report, sending...
I20260806 16:01:37.750929 6579 ts_manager.cc:194] Registered new tserver with Master: cf8644f287b54ee5a074ae67dad8c314 (127.25.31.130:45599)
I20260806 16:01:37.751718 6579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:36111
I20260806 16:01:37.755940 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:01:37.768157 7079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:37.758158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52698 (local address 127.25.31.190:45803)
0806 16:01:37.758547 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:37.758552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:37.758820 (+ 268us) server_negotiation.cc:407] Connection header received
0806 16:01:37.758958 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:37.758962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:37.759012 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:37.759070 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:37.760007 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.760840 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.762053 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.762449 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.762523 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:37.764298 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:37.764302 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:37.764304 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:37.764331 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:37.766139 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:37.766729 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:37.766733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:37.766735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:37.766822 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:37.767144 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:37.767147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:37.767149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:37.767359 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:37.767497 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:37.767846 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 16:01:37.767977 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":87,"threads_started":1}
I20260806 16:01:37.772928 6579 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:01:37.777683 6579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:01:37.815409 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:37.799653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:34583 (local address 127.0.0.1:43084)
0806 16:01:37.803914 (+ 4261us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:37.803966 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:37.804096 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:37.804886 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:37.804892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:37.804916 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:01:37.805053 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:37.805059 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.807609 (+ 2550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.807613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:37.811587 (+ 3974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:37.811598 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.814769 (+ 3171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.814789 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:37.814946 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:37.814994 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 16:01:37.815122 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 16:01:37.815245 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4085,"thread_start_us":96,"threads_started":1}
I20260806 16:01:37.816018 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:37.799820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43084 (local address 127.25.31.129:34583)
0806 16:01:37.803648 (+ 3828us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:37.803654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:37.804232 (+ 578us) server_negotiation.cc:407] Connection header received
0806 16:01:37.804317 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:37.804322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:37.804547 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:37.804681 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:01:37.805973 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.807243 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.811760 (+ 4517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.812649 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.812749 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:37.812951 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:37.815759 (+ 2808us) server_negotiation.cc:299] Negotiation successful
0806 16:01:37.815871 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3730,"thread_start_us":107,"threads_started":1}
I20260806 16:01:37.817888 7089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:37.801626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:45599 (local address 127.0.0.1:48946)
0806 16:01:37.811599 (+ 9973us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:37.811614 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:37.811808 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:37.813276 (+ 1468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:37.813279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:37.813294 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:01:37.813455 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:37.813461 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.814985 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.814988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:37.816134 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:37.816147 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.817428 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.817431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:37.817534 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:37.817561 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 16:01:37.817627 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 16:01:37.817705 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9909,"thread_start_us":96,"threads_started":1}
I20260806 16:01:37.818867 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:37.801791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48946 (local address 127.25.31.130:45599)
0806 16:01:37.803470 (+ 1679us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:37.803475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:37.811734 (+ 8259us) server_negotiation.cc:407] Connection header received
0806 16:01:37.811960 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:37.811965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:37.812155 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:37.813083 (+ 928us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:01:37.813586 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.814843 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.816276 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.817097 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.817185 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:37.817323 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:37.818457 (+ 1134us) server_negotiation.cc:299] Negotiation successful
0806 16:01:37.818624 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1560,"thread_start_us":107,"threads_started":1}
I20260806 16:01:37.819787 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:37.801303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:38757 (local address 127.0.0.1:42368)
0806 16:01:37.807403 (+ 6100us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:37.807419 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:37.809150 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:37.809823 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:37.809826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:37.809844 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:01:37.809934 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:37.809940 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.811487 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.811491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:37.812968 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:37.812981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.816442 (+ 3461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.816447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:37.819455 (+ 3008us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:37.819482 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 16:01:37.819573 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 16:01:37.819650 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3887,"thread_start_us":124,"threads_started":1}
I20260806 16:01:37.820290 7091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:37.802014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42368 (local address 127.25.31.131:38757)
0806 16:01:37.803236 (+ 1222us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:37.803246 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:37.807554 (+ 4308us) server_negotiation.cc:407] Connection header received
0806 16:01:37.809322 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:37.809329 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:37.809544 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:37.809681 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:01:37.810059 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.811218 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.813141 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.814083 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.814205 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:37.814392 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:37.819989 (+ 5597us) server_negotiation.cc:299] Negotiation successful
0806 16:01:37.820146 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":102,"threads_started":1}
I20260806 16:01:37.820107 6714 tablet_service.cc:1467] Processing CreateTablet for tablet 2d302abae387407db4bc52945e05a19a (DEFAULT_TABLE table=test-table [id=e3b60bbac8764d93b3840711f9b56071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:01:37.822513 6846 tablet_service.cc:1467] Processing CreateTablet for tablet 2d302abae387407db4bc52945e05a19a (DEFAULT_TABLE table=test-table [id=e3b60bbac8764d93b3840711f9b56071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:01:37.823556 6846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d302abae387407db4bc52945e05a19a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:01:37.823535 6988 tablet_service.cc:1467] Processing CreateTablet for tablet 2d302abae387407db4bc52945e05a19a (DEFAULT_TABLE table=test-table [id=e3b60bbac8764d93b3840711f9b56071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:01:37.824753 6988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d302abae387407db4bc52945e05a19a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:01:37.830749 6714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d302abae387407db4bc52945e05a19a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:01:37.841252 7092 tablet_bootstrap.cc:492] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314: Bootstrap starting.
I20260806 16:01:37.842005 7093 tablet_bootstrap.cc:492] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e: Bootstrap starting.
I20260806 16:01:37.844493 7092 tablet_bootstrap.cc:654] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314: Neither blocks nor log segments found. Creating new log.
I20260806 16:01:37.844918 7094 tablet_bootstrap.cc:492] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769: Bootstrap starting.
I20260806 16:01:37.845916 7092 log.cc:824] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:37.847021 7093 tablet_bootstrap.cc:654] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e: Neither blocks nor log segments found. Creating new log.
I20260806 16:01:37.848390 7093 log.cc:824] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:37.848474 7094 tablet_bootstrap.cc:654] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769: Neither blocks nor log segments found. Creating new log.
I20260806 16:01:37.850014 7094 log.cc:824] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:37.850343 7092 tablet_bootstrap.cc:492] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314: No bootstrap required, opened a new log
I20260806 16:01:37.850591 7092 ts_tablet_manager.cc:1397] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260806 16:01:37.850999 7093 tablet_bootstrap.cc:492] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e: No bootstrap required, opened a new log
I20260806 16:01:37.851325 7093 ts_tablet_manager.cc:1397] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 16:01:37.856745 7093 raft_consensus.cc:348] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7380626821db4c29abb3716fcda3d769" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34583 } } peers { permanent_uuid: "52ca4f1f8ad34fc4b82104ca1111969e" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38757 } } peers { permanent_uuid: "cf8644f287b54ee5a074ae67dad8c314" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45599 } }
I20260806 16:01:37.857082 7093 raft_consensus.cc:374] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:01:37.857161 7093 raft_consensus.cc:729] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ca4f1f8ad34fc4b82104ca1111969e, State: Initialized, Role: FOLLOWER
I20260806 16:01:37.857160 7092 raft_consensus.cc:348] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7380626821db4c29abb3716fcda3d769" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34583 } } peers { permanent_uuid: "52ca4f1f8ad34fc4b82104ca1111969e" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38757 } } peers { permanent_uuid: "cf8644f287b54ee5a074ae67dad8c314" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45599 } }
I20260806 16:01:37.857414 7092 raft_consensus.cc:374] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:01:37.857475 7092 raft_consensus.cc:729] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf8644f287b54ee5a074ae67dad8c314, State: Initialized, Role: FOLLOWER
I20260806 16:01:37.857843 7093 consensus_queue.cc:260] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7380626821db4c29abb3716fcda3d769" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34583 } } peers { permanent_uuid: "52ca4f1f8ad34fc4b82104ca1111969e" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38757 } } peers { permanent_uuid: "cf8644f287b54ee5a074ae67dad8c314" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45599 } }
I20260806 16:01:37.857874 7092 consensus_queue.cc:260] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7380626821db4c29abb3716fcda3d769" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34583 } } peers { permanent_uuid: "52ca4f1f8ad34fc4b82104ca1111969e" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38757 } } peers { permanent_uuid: "cf8644f287b54ee5a074ae67dad8c314" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45599 } }
I20260806 16:01:37.859223 7092 ts_tablet_manager.cc:1428] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314: Time spent starting tablet: real 0.009s user 0.005s sys 0.002s
I20260806 16:01:37.859223 7093 ts_tablet_manager.cc:1428] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 16:01:37.859604 6920 heartbeater.cc:503] Master 127.25.31.190:45803 was elected leader, sending a full tablet report...
I20260806 16:01:37.859604 7058 heartbeater.cc:503] Master 127.25.31.190:45803 was elected leader, sending a full tablet report...
I20260806 16:01:37.861517 7094 tablet_bootstrap.cc:492] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769: No bootstrap required, opened a new log
I20260806 16:01:37.861725 7094 ts_tablet_manager.cc:1397] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.003s
I20260806 16:01:37.868582 7094 raft_consensus.cc:348] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7380626821db4c29abb3716fcda3d769" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34583 } } peers { permanent_uuid: "52ca4f1f8ad34fc4b82104ca1111969e" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38757 } } peers { permanent_uuid: "cf8644f287b54ee5a074ae67dad8c314" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45599 } }
I20260806 16:01:37.868968 7094 raft_consensus.cc:374] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:01:37.869053 7094 raft_consensus.cc:729] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7380626821db4c29abb3716fcda3d769, State: Initialized, Role: FOLLOWER
I20260806 16:01:37.869581 7094 consensus_queue.cc:260] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7380626821db4c29abb3716fcda3d769" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34583 } } peers { permanent_uuid: "52ca4f1f8ad34fc4b82104ca1111969e" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38757 } } peers { permanent_uuid: "cf8644f287b54ee5a074ae67dad8c314" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45599 } }
I20260806 16:01:37.871011 7094 ts_tablet_manager.cc:1428] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769: Time spent starting tablet: real 0.009s user 0.010s sys 0.001s
I20260806 16:01:37.871443 6782 heartbeater.cc:503] Master 127.25.31.190:45803 was elected leader, sending a full tablet report...
I20260806 16:01:37.900075 7100 raft_consensus.cc:482] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:01:37.900352 7100 raft_consensus.cc:504] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7380626821db4c29abb3716fcda3d769" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34583 } } peers { permanent_uuid: "52ca4f1f8ad34fc4b82104ca1111969e" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38757 } } peers { permanent_uuid: "cf8644f287b54ee5a074ae67dad8c314" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45599 } }
I20260806 16:01:37.903146 7100 leader_election.cc:288] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ca4f1f8ad34fc4b82104ca1111969e (127.25.31.131:38757), cf8644f287b54ee5a074ae67dad8c314 (127.25.31.130:45599)
I20260806 16:01:37.908090 7091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:37.902502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:56939 (local address 127.25.31.131:38757)
0806 16:01:37.902621 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:37.902625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:37.903134 (+ 509us) server_negotiation.cc:407] Connection header received
0806 16:01:37.903328 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:37.903331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:37.903391 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:37.903436 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:01:37.904208 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.904972 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.906773 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.907324 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.907424 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:37.907476 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:37.907885 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 16:01:37.907963 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 16:01:37.908892 7101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:37.902251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:38757 (local address 127.25.31.129:56939)
0806 16:01:37.903043 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:37.903058 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:37.903210 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:37.903571 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:37.903574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:37.903590 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:01:37.904061 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:37.904073 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.905109 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.905112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:37.906637 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:37.906647 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.907568 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.907571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:37.907743 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:37.907765 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 16:01:37.908485 (+ 720us) client_negotiation.cc:240] Negotiation successful
0806 16:01:37.908562 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":499,"threads_started":1}
I20260806 16:01:37.908941 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:37.902923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:43699 (local address 127.25.31.130:45599)
0806 16:01:37.903101 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:37.903106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:37.903738 (+ 632us) server_negotiation.cc:407] Connection header received
0806 16:01:37.903951 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:37.903954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:37.904018 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:37.904071 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:01:37.904777 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.905582 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.907494 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.908120 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.908232 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:37.908279 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:37.908728 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 16:01:37.908815 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 16:01:37.909456 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:37.902269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:45599 (local address 127.25.31.129:43699)
0806 16:01:37.903599 (+ 1330us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:37.903615 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:37.903816 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:37.904236 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:37.904239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:37.904256 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:01:37.904597 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:37.904607 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.905751 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.905754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:37.907334 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:37.907345 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.908382 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.908386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:37.908562 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:37.908587 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 16:01:37.909147 (+ 560us) client_negotiation.cc:240] Negotiation successful
0806 16:01:37.909226 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1249,"mutex_wait_us":13,"thread_start_us":198,"threads_started":1}
I20260806 16:01:37.910424 6873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d302abae387407db4bc52945e05a19a" candidate_uuid: "7380626821db4c29abb3716fcda3d769" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf8644f287b54ee5a074ae67dad8c314" is_pre_election: true
I20260806 16:01:37.910416 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d302abae387407db4bc52945e05a19a" candidate_uuid: "7380626821db4c29abb3716fcda3d769" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ca4f1f8ad34fc4b82104ca1111969e" is_pre_election: true
I20260806 16:01:37.910748 6873 raft_consensus.cc:2454] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7380626821db4c29abb3716fcda3d769 in term 0.
I20260806 16:01:37.910748 7011 raft_consensus.cc:2454] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7380626821db4c29abb3716fcda3d769 in term 0.
I20260806 16:01:37.911378 6667 leader_election.cc:302] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 52ca4f1f8ad34fc4b82104ca1111969e, 7380626821db4c29abb3716fcda3d769, cf8644f287b54ee5a074ae67dad8c314; no voters:
W20260806 16:01:37.911677 6783 tablet.cc:2369] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:01:37.911700 7100 raft_consensus.cc:2781] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:01:37.911761 7100 raft_consensus.cc:482] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:01:37.911800 7100 raft_consensus.cc:3037] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:01:37.913605 7100 raft_consensus.cc:504] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7380626821db4c29abb3716fcda3d769" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34583 } } peers { permanent_uuid: "52ca4f1f8ad34fc4b82104ca1111969e" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38757 } } peers { permanent_uuid: "cf8644f287b54ee5a074ae67dad8c314" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45599 } }
I20260806 16:01:37.913998 7100 leader_election.cc:288] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [CANDIDATE]: Term 1 election: Requested vote from peers 52ca4f1f8ad34fc4b82104ca1111969e (127.25.31.131:38757), cf8644f287b54ee5a074ae67dad8c314 (127.25.31.130:45599)
I20260806 16:01:37.914477 6873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d302abae387407db4bc52945e05a19a" candidate_uuid: "7380626821db4c29abb3716fcda3d769" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf8644f287b54ee5a074ae67dad8c314"
I20260806 16:01:37.914477 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d302abae387407db4bc52945e05a19a" candidate_uuid: "7380626821db4c29abb3716fcda3d769" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ca4f1f8ad34fc4b82104ca1111969e"
I20260806 16:01:37.914601 6873 raft_consensus.cc:3037] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:01:37.914601 7011 raft_consensus.cc:3037] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:01:37.916692 6873 raft_consensus.cc:2454] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7380626821db4c29abb3716fcda3d769 in term 1.
I20260806 16:01:37.916751 7011 raft_consensus.cc:2454] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7380626821db4c29abb3716fcda3d769 in term 1.
I20260806 16:01:37.917021 6667 leader_election.cc:302] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ca4f1f8ad34fc4b82104ca1111969e, 7380626821db4c29abb3716fcda3d769; no voters:
I20260806 16:01:37.917225 7100 raft_consensus.cc:2781] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:01:37.917522 7100 raft_consensus.cc:686] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [term 1 LEADER]: Becoming Leader. State: Replica: 7380626821db4c29abb3716fcda3d769, State: Running, Role: LEADER
I20260806 16:01:37.917860 7100 consensus_queue.cc:237] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7380626821db4c29abb3716fcda3d769" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34583 } } peers { permanent_uuid: "52ca4f1f8ad34fc4b82104ca1111969e" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38757 } } peers { permanent_uuid: "cf8644f287b54ee5a074ae67dad8c314" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45599 } }
I20260806 16:01:37.921205 6579 catalog_manager.cc:5495] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7380626821db4c29abb3716fcda3d769 (127.25.31.129). New cstate: current_term: 1 leader_uuid: "7380626821db4c29abb3716fcda3d769" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7380626821db4c29abb3716fcda3d769" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 34583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52ca4f1f8ad34fc4b82104ca1111969e" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf8644f287b54ee5a074ae67dad8c314" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45599 } health_report { overall_health: UNKNOWN } } }
I20260806 16:01:37.961040 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:37.956667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.25.31.129:34583)
0806 16:01:37.956798 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:37.956802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:37.956913 (+ 111us) server_negotiation.cc:407] Connection header received
0806 16:01:37.957143 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:37.957146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:37.957212 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:37.957325 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:01:37.957604 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.958124 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.959065 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:37.959302 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:37.959430 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:37.959750 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:01:37.960292 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:01:37.960453 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:37.960769 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 16:01:37.960830 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 16:01:37.963078 7059 tablet.cc:2369] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:01:37.974730 7011 raft_consensus.cc:1261] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e [term 1 FOLLOWER]: Refusing update from remote peer 7380626821db4c29abb3716fcda3d769: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:01:37.974730 6873 raft_consensus.cc:1261] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314 [term 1 FOLLOWER]: Refusing update from remote peer 7380626821db4c29abb3716fcda3d769: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:01:37.975495 7104 consensus_queue.cc:1035] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ca4f1f8ad34fc4b82104ca1111969e" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 38757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:01:37.975652 7100 consensus_queue.cc:1035] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf8644f287b54ee5a074ae67dad8c314" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:01:37.994083 7107 mvcc.cc:205] Tried to move back new op lower bound from 7315587473296281600 to 7315587473074417664. Current Snapshot: MvccSnapshot[applied={T|T < 7315587473296281600}]
I20260806 16:01:37.995956 7108 mvcc.cc:205] Tried to move back new op lower bound from 7315587473296281600 to 7315587473074417664. Current Snapshot: MvccSnapshot[applied={T|T < 7315587473296281600}]
I20260806 16:01:38.000581 7111 mvcc.cc:205] Tried to move back new op lower bound from 7315587473296281600 to 7315587473074417664. Current Snapshot: MvccSnapshot[applied={T|T < 7315587473296281600}]
I20260806 16:01:38.013166 6580 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:01:38.013367 6580 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:01:38.016583 6580 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1be1196505e74df4af0f21e3d6975e48: Sending DeleteTablet for 3 replicas of tablet 2d302abae387407db4bc52945e05a19a
I20260806 16:01:38.017547 6988 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d302abae387407db4bc52945e05a19a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:01:38 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:42368
I20260806 16:01:38.017722 6714 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d302abae387407db4bc52945e05a19a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:01:38 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:43084
I20260806 16:01:38.018129 6846 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d302abae387407db4bc52945e05a19a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:01:38 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:48946
I20260806 16:01:38.018672 7120 tablet_replica.cc:330] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314: stopping tablet replica
I20260806 16:01:38.018942 7119 tablet_replica.cc:330] T 2d302abae387407db4bc52945e05a19a P 7380626821db4c29abb3716fcda3d769: stopping tablet replica
I20260806 16:01:38.019053 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 6618
I20260806 16:01:38.021605 7120 raft_consensus.cc:2229] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:01:38.021998 7120 raft_consensus.cc:2258] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:01:38.023593 7120 ts_tablet_manager.cc:1905] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:01:38.029458 7120 ts_tablet_manager.cc:1918] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:01:38.032724 7118 tablet_replica.cc:330] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e: stopping tablet replica
I20260806 16:01:38.033221 7118 raft_consensus.cc:2229] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:01:38.033725 7118 raft_consensus.cc:2258] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:01:38.034080 7120 log.cc:1196] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/2d302abae387407db4bc52945e05a19a
Aug 06 16:01:38 dist-test-slave-tm1g krb5kdc[5558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032097, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for HTTP/127.25.31.148@KRBTEST.COM
I20260806 16:01:38.035898 7118 ts_tablet_manager.cc:1905] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:01:38.038784 7118 ts_tablet_manager.cc:1918] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:01:38.038890 7118 log.cc:1196] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/2d302abae387407db4bc52945e05a19a
I20260806 16:01:38.039228 7118 ts_tablet_manager.cc:1939] T 2d302abae387407db4bc52945e05a19a P 52ca4f1f8ad34fc4b82104ca1111969e: Deleting consensus metadata
I20260806 16:01:38.040652 6566 catalog_manager.cc:4841] TS 52ca4f1f8ad34fc4b82104ca1111969e (127.25.31.131:38757): tablet 2d302abae387407db4bc52945e05a19a (table test-table [id=e3b60bbac8764d93b3840711f9b56071]) successfully deleted
I20260806 16:01:38.040872 7120 ts_tablet_manager.cc:1939] T 2d302abae387407db4bc52945e05a19a P cf8644f287b54ee5a074ae67dad8c314: Deleting consensus metadata
I20260806 16:01:38.041952 6567 catalog_manager.cc:4841] TS cf8644f287b54ee5a074ae67dad8c314 (127.25.31.130:45599): tablet 2d302abae387407db4bc52945e05a19a (table test-table [id=e3b60bbac8764d93b3840711f9b56071]) successfully deleted
W20260806 16:01:38.043440 6564 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:34583 (error 108)
I20260806 16:01:38.044533 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 6786
I20260806 16:01:38.052434 7089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:38.052084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:34583 (local address 127.0.0.1:43104)
0806 16:01:38.052230 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:38.052322 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:34583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 16:01:38.052771 6564 catalog_manager.cc:4584] TS 7380626821db4c29abb3716fcda3d769 (127.25.31.129:34583): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2d302abae387407db4bc52945e05a19a: Network error: Client connection negotiation failed: client connection to 127.25.31.129:34583: connect: Connection refused (error 111)
I20260806 16:01:38.055047 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 6923
I20260806 16:01:38.064191 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 6546
I20260806 16:01:38.074247 25726 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 16:01:38.419194 25726 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 16:01:38.419307 25726 mini_ranger.cc:66] Stopping Ranger...
I20260806 16:01:38.774284 25726 mini_ranger.cc:68] Stopped Ranger
2026-08-06 16:01:38.774 UTC [5589] LOG: received smart shutdown request
2026-08-06 16:01:38.778 UTC [5589] LOG: background worker "logical replication launcher" (PID 5599) exited with exit code 1
2026-08-06 16:01:38.779 UTC [5594] LOG: shutting down
2026-08-06 16:01:38.953 UTC [5589] LOG: database system is shut down
2026-08-06T16:01:38Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50772 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:01:39 dist-test-slave-tm1g krb5kdc[7131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:01:39 dist-test-slave-tm1g krb5kdc[7131](info): set up 2 sockets
Aug 06 16:01:39 dist-test-slave-tm1g krb5kdc[7131](info): commencing operation
krb5kdc: starting...
W20260806 16:01:40.146807 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:01:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:01:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:01:40.264863 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:44761
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:33313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:44761,127.25.31.189:42269,127.25.31.188:43857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:01:40.373123 7145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:40.373356 7145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:40.373395 7145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:40.377341 7145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:01:40.377393 7145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:40.377413 7145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:40.377429 7145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:01:40.377444 7145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:01:40.382560 7145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:33313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:44761,127.25.31.189:42269,127.25.31.188:43857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:44761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:40.383544 7145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:40.384562 7145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:40.392913 7150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:40.393007 7153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:40.393055 7145 server_base.cc:1034] running on GCE node
W20260806 16:01:40.392946 7151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:40.393693 7145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:40.394824 7145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:40.396010 7145 hybrid_clock.cc:648] HybridClock initialized: now 1786032100395988 us; error 40 us; skew 500 ppm
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:40.398895 7145 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:01:40.399827 7145 webserver.cc:460] Webserver started at http://127.25.31.190:34129/ using document root <none> and password file <none>
I20260806 16:01:40.400391 7145 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:40.400484 7145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:40.400720 7145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:01:40.402802 7145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "06c7066084a14b7e8f5527c76ce0b110"
format_stamp: "Formatted at 2026-08-06 16:01:40 on dist-test-slave-tm1g"
server_key: "b4d51048d380b25ed0047cea0c7ec17a"
server_key_iv: "fa064f842da6787de4e9bd7c11984c47"
server_key_version: "encryptionkey@0"
I20260806 16:01:40.403383 7145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "06c7066084a14b7e8f5527c76ce0b110"
format_stamp: "Formatted at 2026-08-06 16:01:40 on dist-test-slave-tm1g"
server_key: "b4d51048d380b25ed0047cea0c7ec17a"
server_key_iv: "fa064f842da6787de4e9bd7c11984c47"
server_key_version: "encryptionkey@0"
I20260806 16:01:40.407341 7145 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:01:40.409771 7160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:40.411114 7145 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:01:40.411264 7145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "06c7066084a14b7e8f5527c76ce0b110"
format_stamp: "Formatted at 2026-08-06 16:01:40 on dist-test-slave-tm1g"
server_key: "b4d51048d380b25ed0047cea0c7ec17a"
server_key_iv: "fa064f842da6787de4e9bd7c11984c47"
server_key_version: "encryptionkey@0"
I20260806 16:01:40.411381 7145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:40.430595 7145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:40.433647 7145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:40.433868 7145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:40.441344 7145 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:44761
I20260806 16:01:40.441367 7213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:44761 every 8 connection(s)
I20260806 16:01:40.442341 7145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:01:40.445070 7214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:01:40.448819 7214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:01:40.450496 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 7145
I20260806 16:01:40.450634 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:01:40.450932 25726 external_mini_cluster.cc:1428] Setting key 9eff3a62f9aa9874fa2e56c02654eb50
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.31.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.189@KRBTEST.COM" created.
I20260806 16:01:40.464617 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.451129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:44761 (local address 127.0.0.1:40714)
0806 16:01:40.451682 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.451718 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:40.452557 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:40.453335 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:40.453342 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:40.453851 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:40.454176 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.454189 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.455992 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.455995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.456727 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.456737 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.457694 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.457697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.457808 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.458422 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:40.458438 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:40.459809 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:40.461585 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.461593 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.461600 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.461822 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.462276 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.462278 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.462279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.462448 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.463033 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:40.463042 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:40.463214 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 16:01:40.463435 (+ 221us) client_negotiation.cc:240] Negotiation successful
0806 16:01:40.463890 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":138,"spinlock_wait_cycles":15616,"thread_start_us":147,"threads_started":1}
I20260806 16:01:40.464622 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.451129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40714 (local address 127.25.31.190:44761)
0806 16:01:40.451532 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:40.451555 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:40.451830 (+ 275us) server_negotiation.cc:407] Connection header received
0806 16:01:40.452937 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:40.452942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:40.453118 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:40.453267 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:40.454366 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.455863 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.456857 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.457510 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.457705 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.459957 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:40.459962 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:40.459972 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:40.459993 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:40.461464 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.461956 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.461960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.461961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.462138 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.462566 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.462568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.462570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.462911 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:40.463016 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:40.463495 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 16:01:40.463803 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":269,"thread_start_us":138,"threads_started":1}
I20260806 16:01:40.467082 7214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:01:40.467813 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.467582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:42269 (local address 127.0.0.1:52780)
0806 16:01:40.467694 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.467759 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 16:01:40.468001 7161 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.31.189:42269: connect: Connection refused (error 111)
I20260806 16:01:40.468770 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.468575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:42269 (local address 127.0.0.1:52796)
0806 16:01:40.468675 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.468723 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 16:01:40.468930 7214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.189:42269: Network error: Client connection negotiation failed: client connection to 127.25.31.189:42269: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.189.keytab.
Entry for principal kudu/127.25.31.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.189.keytab.
I20260806 16:01:40.477279 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.189:42269
--webserver_interface=127.25.31.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.189.keytab
--principal=kudu/127.25.31.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:33313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:44761,127.25.31.189:42269,127.25.31.188:43857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:01:40.518141 7214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } attempt: 1
I20260806 16:01:40.519109 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.518802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:42269 (local address 127.0.0.1:52798)
0806 16:01:40.518964 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.519027 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 16:01:40.519852 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.519628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:42269 (local address 127.0.0.1:52800)
0806 16:01:40.519737 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.519791 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 16:01:40.519985 7214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.189:42269: Network error: Client connection negotiation failed: client connection to 127.25.31.189:42269: connect: Connection refused (error 111)
W20260806 16:01:40.587603 7221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:40.587880 7221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:40.587949 7221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 16:01:40.588147 7214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } attempt: 2
I20260806 16:01:40.588984 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.588654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:42269 (local address 127.0.0.1:52808)
0806 16:01:40.588793 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.588870 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 16:01:40.589763 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.589549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:42269 (local address 127.0.0.1:52818)
0806 16:01:40.589659 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.589711 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:42269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 16:01:40.589898 7214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.189:42269: Network error: Client connection negotiation failed: client connection to 127.25.31.189:42269: connect: Connection refused (error 111)
W20260806 16:01:40.592290 7221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:01:40.592371 7221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:40.592410 7221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:40.592464 7221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:01:40.592502 7221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:01:40.597553 7221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:33313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:44761,127.25.31.189:42269,127.25.31.188:43857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.189.keytab
--principal=kudu/127.25.31.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.189:42269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.31.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:40.598578 7221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:40.599547 7221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:40.607720 7227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:40.607893 7221 server_base.cc:1034] running on GCE node
W20260806 16:01:40.607749 7229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:40.607720 7226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:40.608481 7221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:40.609457 7221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:40.610649 7221 hybrid_clock.cc:648] HybridClock initialized: now 1786032100610610 us; error 40 us; skew 500 ppm
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:40.613385 7221 init.cc:377] Logged in from keytab as kudu/127.25.31.189@KRBTEST.COM (short username kudu)
I20260806 16:01:40.614267 7221 webserver.cc:460] Webserver started at http://127.25.31.189:33985/ using document root <none> and password file <none>
I20260806 16:01:40.614821 7221 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:40.614912 7221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:40.615145 7221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:01:40.617132 7221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/data/instance:
uuid: "1e5c2fc0c45147e79753cc0cae98230f"
format_stamp: "Formatted at 2026-08-06 16:01:40 on dist-test-slave-tm1g"
server_key: "a5ad0d013668829ff627456596f2813e"
server_key_iv: "710115ae16bad946eb7611bc9a9c1be5"
server_key_version: "encryptionkey@0"
I20260806 16:01:40.617699 7221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/wal/instance:
uuid: "1e5c2fc0c45147e79753cc0cae98230f"
format_stamp: "Formatted at 2026-08-06 16:01:40 on dist-test-slave-tm1g"
server_key: "a5ad0d013668829ff627456596f2813e"
server_key_iv: "710115ae16bad946eb7611bc9a9c1be5"
server_key_version: "encryptionkey@0"
I20260806 16:01:40.621316 7221 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 16:01:40.623651 7236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:40.624836 7221 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:01:40.624989 7221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/wal
uuid: "1e5c2fc0c45147e79753cc0cae98230f"
format_stamp: "Formatted at 2026-08-06 16:01:40 on dist-test-slave-tm1g"
server_key: "a5ad0d013668829ff627456596f2813e"
server_key_iv: "710115ae16bad946eb7611bc9a9c1be5"
server_key_version: "encryptionkey@0"
I20260806 16:01:40.625098 7221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:40.658864 7221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:40.661662 7221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:40.661876 7221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:40.669086 7221 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.189:42269
I20260806 16:01:40.669116 7289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.189:42269 every 8 connection(s)
I20260806 16:01:40.669998 7221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/data/info.pb
I20260806 16:01:40.672602 7290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:01:40.673233 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 7221
I20260806 16:01:40.673365 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-1/wal/instance
I20260806 16:01:40.673646 25726 external_mini_cluster.cc:1428] Setting key 8f87272b1c42a8b5dc0d6f4fbcd8ab14
I20260806 16:01:40.676412 7290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:01:40.681067 7214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } attempt: 3
WARNING: no policy specified for kudu/127.25.31.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.188@KRBTEST.COM" created.
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.189@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for kudu/127.25.31.189@KRBTEST.COM
I20260806 16:01:40.692788 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.678292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40724 (local address 127.25.31.190:44761)
0806 16:01:40.678467 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:40.678470 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:40.679447 (+ 977us) server_negotiation.cc:407] Connection header received
0806 16:01:40.680355 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:40.680358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:40.680420 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:40.680522 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:40.681848 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.682700 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.684384 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.684737 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.684846 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.688167 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:40.688170 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:40.688173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:40.688198 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:40.689885 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.690562 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.690565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.690567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.690653 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.691033 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.691036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.691038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.691217 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:40.691334 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:40.692446 (+ 1112us) server_negotiation.cc:299] Negotiation successful
0806 16:01:40.692634 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":87}
I20260806 16:01:40.693202 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.678800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:44761 (local address 127.0.0.1:40724)
0806 16:01:40.679292 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.679339 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:40.680148 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:40.680686 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:40.680692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:40.681039 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:40.681677 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.681696 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.682834 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.682838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.684218 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.684229 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.684903 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.684910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.685282 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.686077 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:40.686128 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:40.688027 (+ 1899us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:40.689991 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.690000 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.690009 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.690449 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.690749 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.690754 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.690759 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.690949 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.691326 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:40.691332 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:40.691575 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 16:01:40.691752 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 16:01:40.692102 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":2,"thread_start_us":144,"threads_started":1}
I20260806 16:01:40.693924 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.681532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:42269 (local address 127.0.0.1:52820)
0806 16:01:40.681704 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.681719 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:40.681797 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:40.682487 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:40.682489 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:40.682630 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:40.682775 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.682785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.684299 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.684304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.685147 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.685159 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.686253 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.686257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.686425 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.687748 (+ 1323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:40.687768 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:40.689579 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:40.692011 (+ 2432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.692018 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.692022 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.692283 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.692682 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.692685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.692687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.692790 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.693396 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:40.693400 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:40.693514 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 16:01:40.693592 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 16:01:40.693738 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":1}
I20260806 16:01:40.694726 7294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.681661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52820 (local address 127.25.31.189:42269)
0806 16:01:40.682040 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:40.682056 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:40.682078 (+ 22us) server_negotiation.cc:407] Connection header received
0806 16:01:40.682117 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:40.682121 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:40.682295 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:40.682477 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:40.682909 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.684131 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.685288 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.686107 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.686677 (+ 570us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.689744 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:40.689753 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:40.689768 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:40.689794 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:40.691868 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.692429 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.692448 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.692453 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.692561 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.692922 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.692927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.692929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.693259 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:40.694233 (+ 974us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:40.694376 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 16:01:40.694531 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":285,"thread_start_us":167,"threads_started":1}
I20260806 16:01:40.694682 7290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:01:40.696408 7214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:01:40.697053 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.696828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:43857 (local address 127.0.0.1:54512)
0806 16:01:40.696936 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.696988 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 16:01:40.697721 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.697524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:43857 (local address 127.0.0.1:54526)
0806 16:01:40.697634 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.697674 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 16:01:40.697940 7214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:43857: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.189@KRBTEST.COM for kudu/127.25.31.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.188.keytab.
Entry for principal kudu/127.25.31.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.188.keytab.
I20260806 16:01:40.701556 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.188:43857
--webserver_interface=127.25.31.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.188.keytab
--principal=kudu/127.25.31.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:33313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:44761,127.25.31.189:42269,127.25.31.188:43857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:01:40.703729 7294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.695485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52824 (local address 127.25.31.189:42269)
0806 16:01:40.695695 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:40.695699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:40.695711 (+ 12us) server_negotiation.cc:407] Connection header received
0806 16:01:40.695742 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:40.695744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:40.695781 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:40.695862 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:40.696278 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.696910 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.697991 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.698362 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.698526 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.700371 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:40.700374 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:40.700375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:40.700396 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:40.702006 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.702433 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.702436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.702440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.702529 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.702787 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.702793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.702798 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.703048 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:40.703130 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:40.703467 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 16:01:40.703609 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":158,"thread_start_us":82,"threads_started":1}
I20260806 16:01:40.703778 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.695255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:42269 (local address 127.0.0.1:52824)
0806 16:01:40.695362 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.695376 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:40.695447 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:40.695904 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:40.695909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:40.696075 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:40.696166 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.696171 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.697014 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.697020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.697861 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.697874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.698471 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.698474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.698583 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.698992 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:40.699002 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:40.700256 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:40.702112 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.702114 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.702116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.702294 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.702623 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.702625 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.702626 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.702699 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.703151 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:40.703156 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:40.703267 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 16:01:40.703409 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 16:01:40.703561 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":19}
I20260806 16:01:40.704454 7290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:01:40.705114 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.704897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:43857 (local address 127.0.0.1:54542)
0806 16:01:40.705004 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.705068 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 16:01:40.705646 7240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
I20260806 16:01:40.706547 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.706355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:43857 (local address 127.0.0.1:54558)
0806 16:01:40.706449 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.706502 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 16:01:40.706707 7290 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:43857: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
I20260806 16:01:40.729107 7214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } attempt: 1
I20260806 16:01:40.729875 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.729609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:43857 (local address 127.0.0.1:54572)
0806 16:01:40.729744 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.729799 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 16:01:40.730610 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.730413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:43857 (local address 127.0.0.1:54576)
0806 16:01:40.730505 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.730559 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 16:01:40.730729 7214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:43857: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
I20260806 16:01:40.755940 7290 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } attempt: 1
I20260806 16:01:40.756762 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.756470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:43857 (local address 127.0.0.1:54590)
0806 16:01:40.756623 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.756688 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 16:01:40.757597 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.757338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:43857 (local address 127.0.0.1:54602)
0806 16:01:40.757487 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.757542 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260806 16:01:40.757735 7290 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:43857: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
I20260806 16:01:40.805904 7214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } attempt: 2
I20260806 16:01:40.806777 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.806483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:43857 (local address 127.0.0.1:54606)
0806 16:01:40.806639 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.806700 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 16:01:40.807508 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.807301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:43857 (local address 127.0.0.1:54608)
0806 16:01:40.807405 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.807458 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 16:01:40.807626 7214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:43857: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
W20260806 16:01:40.813683 7297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:40.813946 7297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:40.814049 7297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:40.818183 7297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:01:40.818259 7297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:40.818331 7297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:40.818392 7297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:01:40.818428 7297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:01:40.823487 7297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:33313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:44761,127.25.31.189:42269,127.25.31.188:43857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.188.keytab
--principal=kudu/127.25.31.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.188:43857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.31.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:40.824446 7297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:40.825409 7297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:01:40.825892 7290 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } attempt: 2
I20260806 16:01:40.826655 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.826390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:43857 (local address 127.0.0.1:54618)
0806 16:01:40.826523 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.826590 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 16:01:40.827389 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.827162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:43857 (local address 127.0.0.1:54630)
0806 16:01:40.827289 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.827340 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 16:01:40.827517 7290 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:43857: Network error: Client connection negotiation failed: client connection to 127.25.31.188:43857: connect: Connection refused (error 111)
W20260806 16:01:40.834049 7303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:40.834049 7307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:40.834044 7304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:40.834285 7297 server_base.cc:1034] running on GCE node
I20260806 16:01:40.834980 7297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:40.836014 7297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:40.837221 7297 hybrid_clock.cc:648] HybridClock initialized: now 1786032100837207 us; error 24 us; skew 500 ppm
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:40.840167 7297 init.cc:377] Logged in from keytab as kudu/127.25.31.188@KRBTEST.COM (short username kudu)
I20260806 16:01:40.840982 7297 webserver.cc:460] Webserver started at http://127.25.31.188:44583/ using document root <none> and password file <none>
I20260806 16:01:40.841492 7297 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:40.841550 7297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:40.841722 7297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:01:40.843757 7297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/data/instance:
uuid: "23549723199145e7ad676c8d11f2e82d"
format_stamp: "Formatted at 2026-08-06 16:01:40 on dist-test-slave-tm1g"
server_key: "aa9e714f9de0fa654c4c7d3c8d158530"
server_key_iv: "2564ed217e054861cb53a250e4b7f1e7"
server_key_version: "encryptionkey@0"
I20260806 16:01:40.844262 7297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/wal/instance:
uuid: "23549723199145e7ad676c8d11f2e82d"
format_stamp: "Formatted at 2026-08-06 16:01:40 on dist-test-slave-tm1g"
server_key: "aa9e714f9de0fa654c4c7d3c8d158530"
server_key_iv: "2564ed217e054861cb53a250e4b7f1e7"
server_key_version: "encryptionkey@0"
I20260806 16:01:40.847878 7297 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 16:01:40.850150 7314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:40.851392 7297 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:01:40.851521 7297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/wal
uuid: "23549723199145e7ad676c8d11f2e82d"
format_stamp: "Formatted at 2026-08-06 16:01:40 on dist-test-slave-tm1g"
server_key: "aa9e714f9de0fa654c4c7d3c8d158530"
server_key_iv: "2564ed217e054861cb53a250e4b7f1e7"
server_key_version: "encryptionkey@0"
I20260806 16:01:40.851624 7297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:40.865798 7297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:40.869072 7297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:40.869267 7297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:40.876299 7297 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.188:43857
I20260806 16:01:40.876322 7367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.188:43857 every 8 connection(s)
I20260806 16:01:40.877209 7297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/data/info.pb
I20260806 16:01:40.878188 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 7297
I20260806 16:01:40.878278 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786031939584994-25726-0/minicluster-data/master-2/wal/instance
I20260806 16:01:40.878602 25726 external_mini_cluster.cc:1428] Setting key 80b45b65b7cad04f66665716a73faf1a
I20260806 16:01:40.880141 7368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:01:40.883764 7368 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.188@KRBTEST.COM
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.188@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:01:40.892051 7371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.879959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54646 (local address 127.25.31.188:43857)
0806 16:01:40.880410 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:40.880429 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:40.880462 (+ 33us) server_negotiation.cc:407] Connection header received
0806 16:01:40.881078 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:40.881092 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:40.881402 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:40.881722 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:40.882603 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.883566 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.884494 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.884980 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.885138 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.887765 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:40.887770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:40.887777 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:40.887802 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:40.889344 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.889783 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.889793 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.889797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.889907 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.890192 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.890197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.890199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.890623 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:40.890839 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:40.891127 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 16:01:40.891392 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":138,"threads_started":1}
I20260806 16:01:40.894797 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.885441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.25.31.190:44761)
0806 16:01:40.885579 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:40.885584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:40.885918 (+ 334us) server_negotiation.cc:407] Connection header received
0806 16:01:40.886082 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:40.886085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:40.886133 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:40.886228 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:40.886918 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.887488 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.888716 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.888931 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.889043 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.891198 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:40.891201 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:40.891203 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:40.891253 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:40.892883 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.893340 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.893345 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.893348 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.893439 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.893752 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.893754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.893756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.894022 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:40.894117 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:40.894538 (+ 421us) server_negotiation.cc:299] Negotiation successful
0806 16:01:40.894648 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 16:01:40.894810 7372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.885379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:44761 (local address 127.0.0.1:40740)
0806 16:01:40.885789 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.885826 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:40.885968 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:40.886277 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:40.886282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:40.886659 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:40.886786 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.886792 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.887603 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.887608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.888403 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.888413 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.889056 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.889059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.889192 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.889678 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:40.889689 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:40.891046 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:40.892999 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.893010 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.893015 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.893230 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.893532 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.893537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.893539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.893644 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.894178 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:40.894189 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:40.894280 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 16:01:40.894479 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 16:01:40.894648 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":134,"threads_started":1}
I20260806 16:01:40.895814 7368 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.188@KRBTEST.COM for kudu/127.25.31.189@KRBTEST.COM
I20260806 16:01:40.903470 7373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.894110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40744 (local address 127.25.31.190:44761)
0806 16:01:40.894490 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:40.894500 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:40.894513 (+ 13us) server_negotiation.cc:407] Connection header received
0806 16:01:40.894592 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:40.894595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:40.894639 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:40.894728 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:40.895818 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.896489 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.897246 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.897514 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.897625 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.899913 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:40.899915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:40.899917 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:40.899937 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:40.901717 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.902118 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.902124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.902129 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.902219 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.902534 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.902536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.902537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.902764 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:40.902923 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:40.903145 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 16:01:40.903262 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":127,"threads_started":1}
I20260806 16:01:40.904040 7372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.896246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:42269 (local address 127.0.0.1:52830)
0806 16:01:40.896349 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.896365 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:40.896479 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:40.896756 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:40.896759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:40.896919 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:40.896997 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.897003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.897707 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.897710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.898250 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.898258 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.898775 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.898779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.898871 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.899293 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:40.899306 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:40.900481 (+ 1175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:40.902197 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.902200 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.902201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.902516 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.902791 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.902793 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.902794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.902908 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.903379 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:40.903384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:40.903467 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 16:01:40.903806 (+ 339us) client_negotiation.cc:240] Negotiation successful
0806 16:01:40.903911 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 16:01:40.904314 7294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.896405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52830 (local address 127.25.31.189:42269)
0806 16:01:40.896560 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:40.896565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:40.896585 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:01:40.896621 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:40.896624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:40.896659 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:40.896744 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:40.897113 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.897594 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.898417 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.898655 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.898763 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.900618 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:40.900623 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:40.900627 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:40.900662 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:40.902095 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.902621 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.902625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.902626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.902702 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.903015 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.903017 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.903019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.903265 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:40.903364 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:40.903889 (+ 525us) server_negotiation.cc:299] Negotiation successful
0806 16:01:40.904006 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 16:01:40.904914 7368 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:01:40.906769 7214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } attempt: 3
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.188@KRBTEST.COM for kudu/127.25.31.188@KRBTEST.COM
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.189@KRBTEST.COM
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for kudu/127.25.31.188@KRBTEST.COM
I20260806 16:01:40.914201 7371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.905454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54650 (local address 127.25.31.188:43857)
0806 16:01:40.905568 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:40.905571 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:40.905610 (+ 39us) server_negotiation.cc:407] Connection header received
0806 16:01:40.905725 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:40.905727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:40.905758 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:40.905856 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:40.906182 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.906899 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.907876 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.908245 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.908370 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.910719 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:40.910722 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:40.910724 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:40.910744 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:40.912475 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.912872 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.912877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.912882 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.912959 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.913247 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.913250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.913251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.913460 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:40.913531 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:40.913852 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 16:01:40.914014 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":40}
I20260806 16:01:40.914759 7372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.905386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:43857 (local address 127.0.0.1:54650)
0806 16:01:40.905555 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.905568 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:40.905645 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:40.905873 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:40.905876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:40.906023 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:40.906078 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.906083 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.907035 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.907038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.907769 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.907778 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.908371 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.908374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.908454 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.908883 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:40.908903 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:40.910600 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:40.912591 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.912594 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.912596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.912779 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.913053 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.913055 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.913057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.913153 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.913591 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:40.913593 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:40.913685 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 16:01:40.914454 (+ 769us) client_negotiation.cc:240] Negotiation successful
0806 16:01:40.914579 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260806 16:01:40.915225 7294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.904691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52844 (local address 127.25.31.189:42269)
0806 16:01:40.904880 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:40.904885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:40.904908 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:01:40.904942 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:40.904945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:40.904978 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:40.905069 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:40.905795 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.906258 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.907194 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.907541 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.907639 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.911259 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:40.911264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:40.911269 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:40.911296 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:40.912994 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.913497 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.913502 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.913508 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.913595 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.913924 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.913929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.913930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.914144 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:40.914433 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:40.914847 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 16:01:40.914959 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260806 16:01:40.917804 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.907109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:43857 (local address 127.0.0.1:54656)
0806 16:01:40.907273 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.907283 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:40.907333 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:40.908032 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:40.908035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:40.908194 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:40.908261 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.908265 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.909863 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.909866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.910726 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.910736 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.911300 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.911304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.911410 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.912098 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:40.912109 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:40.913580 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:40.916425 (+ 2845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.916428 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.916429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.916615 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.916976 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.916978 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.916980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.917055 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.917448 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:40.917450 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:40.917525 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 16:01:40.917601 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 16:01:40.917698 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260806 16:01:40.917950 7374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.907536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54656 (local address 127.25.31.188:43857)
0806 16:01:40.907824 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:40.907834 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:40.907848 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:01:40.907884 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:40.907890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:40.907938 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:40.908016 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:40.908380 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.909718 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.910830 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.911185 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.911307 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.913704 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:40.913707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:40.913710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:40.913733 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:40.916307 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.916711 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.916714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.916716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.916829 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.917152 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.917156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.917157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.917352 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:40.917451 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:40.917674 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 16:01:40.917787 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":227,"thread_start_us":119,"threads_started":1}
I20260806 16:01:40.918260 7368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d: Bootstrap starting.
I20260806 16:01:40.918648 7290 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } attempt: 3
I20260806 16:01:40.921001 7214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110: Bootstrap starting.
I20260806 16:01:40.921200 7368 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d: Neither blocks nor log segments found. Creating new log.
I20260806 16:01:40.921243 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:01:40.922071 7368 log.cc:824] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:40.923556 7214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110: Neither blocks nor log segments found. Creating new log.
Aug 06 16:01:40 dist-test-slave-tm1g krb5kdc[7131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032100, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.189@KRBTEST.COM for kudu/127.25.31.188@KRBTEST.COM
I20260806 16:01:40.924358 7214 log.cc:824] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:40.924376 7368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d: No bootstrap required, opened a new log
I20260806 16:01:40.927461 7214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110: No bootstrap required, opened a new log
I20260806 16:01:40.928279 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.918988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:43857 (local address 127.0.0.1:54658)
0806 16:01:40.919195 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:40.919209 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:40.919277 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:40.919725 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:40.919729 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:40.919923 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:40.920022 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.920029 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.921044 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.921051 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.921854 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:40.921866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.922504 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.922510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:40.922601 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.923055 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:40.923073 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:40.924486 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:40.926549 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.926554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.926559 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.926826 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.927169 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:40.927173 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:40.927175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:40.927270 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:40.927800 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:40.927804 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:40.927894 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 16:01:40.927972 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 16:01:40.928091 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260806 16:01:40.928571 7368 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } }
I20260806 16:01:40.928910 7368 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:01:40.929010 7368 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23549723199145e7ad676c8d11f2e82d, State: Initialized, Role: FOLLOWER
I20260806 16:01:40.929117 7374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.919400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54658 (local address 127.25.31.188:43857)
0806 16:01:40.919494 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:40.919498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:40.919509 (+ 11us) server_negotiation.cc:407] Connection header received
0806 16:01:40.919544 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:40.919549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:40.919595 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:40.920128 (+ 533us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:40.920204 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.920907 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.922005 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.922355 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.922807 (+ 452us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.924608 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:40.924612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:40.924616 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:40.924641 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:40.926430 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.926970 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.926975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.926977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.927060 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.927400 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.927405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.927408 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.927671 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:40.928770 (+ 1099us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:40.928846 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 16:01:40.928950 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 16:01:40.929585 7368 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } }
I20260806 16:01:40.931212 7368 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:01:40.931627 7214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } }
I20260806 16:01:40.931883 7214 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:01:40.931957 7214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06c7066084a14b7e8f5527c76ce0b110, State: Initialized, Role: FOLLOWER
I20260806 16:01:40.932407 7214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } }
I20260806 16:01:40.933537 7290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f: Bootstrap starting.
I20260806 16:01:40.933779 7214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 16:01:40.935573 7166 tablet.cc:2369] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:01:40.936414 7290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f: Neither blocks nor log segments found. Creating new log.
W20260806 16:01:40.937218 7396 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:01:40.937300 7396 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:01:40.937463 7290 log.cc:824] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:40.937587 7390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } } }
I20260806 16:01:40.937714 7390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:01:40.937956 7384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } } }
I20260806 16:01:40.938063 7384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:01:40.936309 7297 master_runner.cc:186] Error getting master registration for 127.25.31.190:44761: OK, instance_id { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" instance_seqno: 1786032100439247 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:01:40.939677 7290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f: No bootstrap required, opened a new log
I20260806 16:01:40.939805 7297 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:01:40.935606 7373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.924878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40752 (local address 127.25.31.190:44761)
0806 16:01:40.924981 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:40.924985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:40.926006 (+ 1021us) server_negotiation.cc:407] Connection header received
0806 16:01:40.926197 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:40.926201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:40.926254 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:40.926344 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:40.927363 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.928132 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.929389 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.929718 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.929794 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.931746 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:40.931750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:40.931753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:40.931779 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:40.933477 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.934167 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.934171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.934172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.934259 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.934633 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.934636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.934638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.934861 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:40.934969 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:40.935335 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 16:01:40.935453 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 16:01:40.945195 7409 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 16:01:40.945293 7294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.925058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52854 (local address 127.25.31.189:42269)
0806 16:01:40.925171 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:40.925177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:40.936020 (+ 10843us) server_negotiation.cc:407] Connection header received
0806 16:01:40.936187 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:40.936190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:40.936238 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:40.936291 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:40.937181 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.937912 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.939050 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.939344 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.939419 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.941727 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:40.941731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:40.941734 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:40.941759 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:40.943419 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.943911 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.943916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.943920 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.944007 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.944326 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.944329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.944332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.944566 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:40.944672 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:40.945028 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 16:01:40.945145 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 16:01:40.945261 7145 master_runner.cc:186] Error getting master registration for 127.25.31.189:42269: OK, instance_id { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" instance_seqno: 1786032100667115 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:01:40.947988 7145 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:01:40.949862 7290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } }
I20260806 16:01:40.950066 7290 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:01:40.950129 7290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e5c2fc0c45147e79753cc0cae98230f, State: Initialized, Role: FOLLOWER
I20260806 16:01:40.950655 7290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } }
W20260806 16:01:40.945276 7409 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:01:40.952119 7290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:01:40.955631 7371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:40.925501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54674 (local address 127.25.31.188:43857)
0806 16:01:40.925623 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:40.925628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:40.938399 (+ 12771us) server_negotiation.cc:407] Connection header received
0806 16:01:40.938566 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:40.938569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:40.938612 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:40.938663 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:40.939556 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.940218 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.941417 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:40.941760 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:40.941850 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:40.951419 (+ 9569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:40.951423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:40.951426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:40.951454 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:40.953169 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.953850 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.953854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.953856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.953944 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:40.954262 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:40.954266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:40.954268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:40.954544 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:40.954667 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:40.955354 (+ 687us) server_negotiation.cc:299] Negotiation successful
0806 16:01:40.955491 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 16:01:40.960634 7221 master_runner.cc:263] No leader master found from master 1e5c2fc0c45147e79753cc0cae98230f
I20260806 16:01:40.960714 7221 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 16:01:40.962385 7423 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:01:40.962524 7423 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:01:40.962733 7411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } } }
I20260806 16:01:40.962841 7411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:01:40.991560 7390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:01:40.991746 7390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } }
I20260806 16:01:40.992743 7390 leader_election.cc:288] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e5c2fc0c45147e79753cc0cae98230f (127.25.31.189:42269), 23549723199145e7ad676c8d11f2e82d (127.25.31.188:43857)
I20260806 16:01:40.993482 7342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "06c7066084a14b7e8f5527c76ce0b110" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23549723199145e7ad676c8d11f2e82d" is_pre_election: true
I20260806 16:01:40.993783 7342 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06c7066084a14b7e8f5527c76ce0b110 in term 0.
I20260806 16:01:40.994199 7164 leader_election.cc:302] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06c7066084a14b7e8f5527c76ce0b110, 23549723199145e7ad676c8d11f2e82d; no voters:
I20260806 16:01:40.995081 7263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "06c7066084a14b7e8f5527c76ce0b110" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e5c2fc0c45147e79753cc0cae98230f" is_pre_election: true
I20260806 16:01:40.995326 7263 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06c7066084a14b7e8f5527c76ce0b110 in term 0.
I20260806 16:01:40.995987 7390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:01:40.996070 7390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:01:40.996112 7390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:01:40.997403 7390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } }
I20260806 16:01:40.997726 7390 leader_election.cc:288] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [CANDIDATE]: Term 1 election: Requested vote from peers 1e5c2fc0c45147e79753cc0cae98230f (127.25.31.189:42269), 23549723199145e7ad676c8d11f2e82d (127.25.31.188:43857)
I20260806 16:01:40.998442 7263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "06c7066084a14b7e8f5527c76ce0b110" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e5c2fc0c45147e79753cc0cae98230f"
I20260806 16:01:40.998565 7263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:01:40.999854 7263 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06c7066084a14b7e8f5527c76ce0b110 in term 1.
I20260806 16:01:41.000162 7161 leader_election.cc:302] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06c7066084a14b7e8f5527c76ce0b110, 1e5c2fc0c45147e79753cc0cae98230f; no voters:
I20260806 16:01:41.000409 7342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "06c7066084a14b7e8f5527c76ce0b110" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23549723199145e7ad676c8d11f2e82d"
I20260806 16:01:41.000530 7342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:01:41.001726 7342 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06c7066084a14b7e8f5527c76ce0b110 in term 1.
I20260806 16:01:41.004546 7390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:01:41.004818 7390 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [term 1 LEADER]: Becoming Leader. State: Replica: 06c7066084a14b7e8f5527c76ce0b110, State: Running, Role: LEADER
I20260806 16:01:41.005210 7390 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } }
I20260806 16:01:41.007068 7390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06c7066084a14b7e8f5527c76ce0b110. Latest consensus state: current_term: 1 leader_uuid: "06c7066084a14b7e8f5527c76ce0b110" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } } }
I20260806 16:01:41.007231 7390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [sys.catalog]: This master's current role is: LEADER
I20260806 16:01:41.007661 7427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:01:41.008652 7427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:01:41.018486 7342 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d [term 1 FOLLOWER]: Refusing update from remote peer 06c7066084a14b7e8f5527c76ce0b110: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:01:41.019269 7425 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:01:41.023723 7384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06c7066084a14b7e8f5527c76ce0b110. Latest consensus state: current_term: 1 leader_uuid: "06c7066084a14b7e8f5527c76ce0b110" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } } }
I20260806 16:01:41.023844 7384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:01:41.024933 7263 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f [term 1 FOLLOWER]: Refusing update from remote peer 06c7066084a14b7e8f5527c76ce0b110: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:01:41.025558 7425 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:01:41.032689 7411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06c7066084a14b7e8f5527c76ce0b110. Latest consensus state: current_term: 1 leader_uuid: "06c7066084a14b7e8f5527c76ce0b110" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } } }
I20260806 16:01:41.032814 7411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:01:41.038225 7425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "06c7066084a14b7e8f5527c76ce0b110" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } } }
I20260806 16:01:41.038378 7425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [sys.catalog]: This master's current role is: LEADER
I20260806 16:01:41.039570 7411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "06c7066084a14b7e8f5527c76ce0b110" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } } }
I20260806 16:01:41.039693 7411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:01:41.039894 7428 mvcc.cc:205] Tried to move back new op lower bound from 7315587485764009984 to 7315587485720788992. Current Snapshot: MvccSnapshot[applied={T|T < 7315587485764009984}]
I20260806 16:01:41.040330 7431 mvcc.cc:205] Tried to move back new op lower bound from 7315587485764009984 to 7315587485720788992. Current Snapshot: MvccSnapshot[applied={T|T < 7315587485764009984}]
I20260806 16:01:41.044600 7390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "06c7066084a14b7e8f5527c76ce0b110" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } } }
I20260806 16:01:41.044713 7390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110 [sys.catalog]: This master's current role is: LEADER
I20260806 16:01:41.045876 7384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "06c7066084a14b7e8f5527c76ce0b110" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06c7066084a14b7e8f5527c76ce0b110" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 44761 } } peers { permanent_uuid: "1e5c2fc0c45147e79753cc0cae98230f" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 42269 } } peers { permanent_uuid: "23549723199145e7ad676c8d11f2e82d" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 43857 } } }
I20260806 16:01:41.045986 7384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:01:41.046519 7429 mvcc.cc:205] Tried to move back new op lower bound from 7315587485764009984 to 7315587485720788992. Current Snapshot: MvccSnapshot[applied={T|T < 7315587485764009984}]
I20260806 16:01:41.050134 7427 catalog_manager.cc:1347] Generated new cluster ID: ff761ead9a85452f99855a64d437e418
I20260806 16:01:41.050202 7427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:01:41.086273 7427 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:01:41.087774 7427 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:01:41.105175 7427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06c7066084a14b7e8f5527c76ce0b110: Generated new TSK 0
I20260806 16:01:41.106225 7427 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:01:41.938654 7396 catalog_manager.cc:1259] Loaded cluster ID: ff761ead9a85452f99855a64d437e418
I20260806 16:01:41.938740 7396 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d: loading cluster ID for follower catalog manager: success
I20260806 16:01:41.940325 7396 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d: acquiring CA information for follower catalog manager: success
I20260806 16:01:41.941540 7396 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 23549723199145e7ad676c8d11f2e82d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:01:41.963740 7423 catalog_manager.cc:1259] Loaded cluster ID: ff761ead9a85452f99855a64d437e418
I20260806 16:01:41.963845 7423 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f: loading cluster ID for follower catalog manager: success
I20260806 16:01:41.965452 7423 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f: acquiring CA information for follower catalog manager: success
I20260806 16:01:41.966625 7423 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e5c2fc0c45147e79753cc0cae98230f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:01:42.168821 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 7145
I20260806 16:01:42.177268 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 7221
I20260806 16:01:42.183898 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 7297
2026-08-06T16:01:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3097 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:01:42 dist-test-slave-tm1g krb5kdc[7443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:01:42 dist-test-slave-tm1g krb5kdc[7443](info): set up 2 sockets
Aug 06 16:01:42 dist-test-slave-tm1g krb5kdc[7443](info): commencing operation
krb5kdc: starting...
W20260806 16:01:43.238817 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:01:43 dist-test-slave-tm1g krb5kdc[7443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:01:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:01:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:01:43.357796 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42417
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:42417
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:01:43.465109 7457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:43.465401 7457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:43.465476 7457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:43.469404 7457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:01:43.469491 7457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:43.469533 7457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:43.469591 7457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:01:43.469630 7457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:01:43.474653 7457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:42417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:43.475623 7457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:43.476581 7457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:43.485590 7465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:43.485616 7463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:43.485613 7462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:43.485662 7457 server_base.cc:1034] running on GCE node
I20260806 16:01:43.486456 7457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:43.487408 7457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:43.488576 7457 hybrid_clock.cc:648] HybridClock initialized: now 1786032103488558 us; error 30 us; skew 500 ppm
Aug 06 16:01:43 dist-test-slave-tm1g krb5kdc[7443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032103, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:43.491242 7457 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:01:43.492165 7457 webserver.cc:460] Webserver started at http://127.25.31.190:40807/ using document root <none> and password file <none>
I20260806 16:01:43.492702 7457 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:43.492800 7457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:43.493000 7457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:01:43.494953 7457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "8303cc2a529f4d9687138d2fd569be4a"
format_stamp: "Formatted at 2026-08-06 16:01:43 on dist-test-slave-tm1g"
server_key: "4bc61ad0c5614a626c5ac0d613d9cd39"
server_key_iv: "705f3a60ffb16864691cea8242846962"
server_key_version: "encryptionkey@0"
I20260806 16:01:43.495436 7457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "8303cc2a529f4d9687138d2fd569be4a"
format_stamp: "Formatted at 2026-08-06 16:01:43 on dist-test-slave-tm1g"
server_key: "4bc61ad0c5614a626c5ac0d613d9cd39"
server_key_iv: "705f3a60ffb16864691cea8242846962"
server_key_version: "encryptionkey@0"
I20260806 16:01:43.499054 7457 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 16:01:43.501356 7472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:43.502671 7457 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 16:01:43.502796 7457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "8303cc2a529f4d9687138d2fd569be4a"
format_stamp: "Formatted at 2026-08-06 16:01:43 on dist-test-slave-tm1g"
server_key: "4bc61ad0c5614a626c5ac0d613d9cd39"
server_key_iv: "705f3a60ffb16864691cea8242846962"
server_key_version: "encryptionkey@0"
I20260806 16:01:43.502941 7457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:43.536019 7457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:43.538803 7457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:43.539029 7457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:43.546157 7457 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:42417
I20260806 16:01:43.546191 7525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:42417 every 8 connection(s)
I20260806 16:01:43.547114 7457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:01:43.549721 7526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:01:43.553751 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 7457
I20260806 16:01:43.553889 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:01:43.554175 25726 external_mini_cluster.cc:1428] Setting key 61ec30faef4b60484670eafc39f3e713
I20260806 16:01:43.555230 7526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a: Bootstrap starting.
I20260806 16:01:43.557412 7526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a: Neither blocks nor log segments found. Creating new log.
I20260806 16:01:43.558346 7526 log.cc:824] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:43.560453 7526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a: No bootstrap required, opened a new log
Aug 06 16:01:43 dist-test-slave-tm1g krb5kdc[7443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:01:43.564632 7526 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8303cc2a529f4d9687138d2fd569be4a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42417 } }
I20260806 16:01:43.564872 7526 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:01:43.564952 7526 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8303cc2a529f4d9687138d2fd569be4a, State: Initialized, Role: FOLLOWER
I20260806 16:01:43.565434 7526 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8303cc2a529f4d9687138d2fd569be4a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42417 } }
I20260806 16:01:43.565603 7526 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:01:43.565698 7526 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:01:43.565795 7526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:01:43.566717 7526 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8303cc2a529f4d9687138d2fd569be4a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42417 } }
I20260806 16:01:43.567045 7526 leader_election.cc:302] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8303cc2a529f4d9687138d2fd569be4a; no voters:
I20260806 16:01:43.567359 7526 leader_election.cc:288] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:01:43.567435 7531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:01:43.567627 7531 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [term 1 LEADER]: Becoming Leader. State: Replica: 8303cc2a529f4d9687138d2fd569be4a, State: Running, Role: LEADER
I20260806 16:01:43.567970 7531 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8303cc2a529f4d9687138d2fd569be4a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42417 } }
I20260806 16:01:43.568591 7526 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:01:43.569108 7531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8303cc2a529f4d9687138d2fd569be4a. Latest consensus state: current_term: 1 leader_uuid: "8303cc2a529f4d9687138d2fd569be4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8303cc2a529f4d9687138d2fd569be4a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42417 } } }
I20260806 16:01:43.569283 7531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [sys.catalog]: This master's current role is: LEADER
I20260806 16:01:43.569518 7532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8303cc2a529f4d9687138d2fd569be4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8303cc2a529f4d9687138d2fd569be4a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42417 } } }
I20260806 16:01:43.569680 7532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a [sys.catalog]: This master's current role is: LEADER
I20260806 16:01:43.569751 7538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:01:43.569967 7529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:43.555542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40596 (local address 127.25.31.190:42417)
0806 16:01:43.555929 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:43.555953 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:43.555987 (+ 34us) server_negotiation.cc:407] Connection header received
0806 16:01:43.556590 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:43.556606 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:43.556924 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:43.557183 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:43.558165 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:43.559605 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:43.560728 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:43.561183 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:43.561295 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:43.564059 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:43.564064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:43.564077 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:43.564103 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:43.566104 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:43.566600 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:43.566606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:43.566612 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:43.566731 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:43.567082 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:43.567085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:43.567087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:43.567646 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:43.568470 (+ 824us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:43.568587 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 16:01:43.568889 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":132,"threads_started":1}
I20260806 16:01:43.573110 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:01:43.573558 7538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:01:43.579164 7538 catalog_manager.cc:1347] Generated new cluster ID: d48f2cfaac274a249b70cf6b242e7852
I20260806 16:01:43.579229 7538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:01:43.596675 7538 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:01:43.597532 7538 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:01:43.606073 7538 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8303cc2a529f4d9687138d2fd569be4a: Generated new TSK 0
I20260806 16:01:43.606775 7538 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:01:43.628887 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 7457
2026-08-06T16:01:43Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1443 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:01:43 dist-test-slave-tm1g krb5kdc[7553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:01:43 dist-test-slave-tm1g krb5kdc[7553](info): set up 2 sockets
Aug 06 16:01:43 dist-test-slave-tm1g krb5kdc[7553](info): commencing operation
krb5kdc: starting...
W20260806 16:01:44.675004 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:01:44 dist-test-slave-tm1g krb5kdc[7553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:01:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:01:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:01:44.793546 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42995
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:42995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:01:44.901966 7567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:44.902277 7567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:44.902379 7567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:44.906482 7567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:01:44.906574 7567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:44.906613 7567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:44.906672 7567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:01:44.906710 7567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:01:44.911947 7567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:42995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:44.912997 7567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:44.914019 7567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:44.922371 7575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:44.922333 7573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:44.922351 7572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:44.922559 7567 server_base.cc:1034] running on GCE node
I20260806 16:01:44.923185 7567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:44.924247 7567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:44.925509 7567 hybrid_clock.cc:648] HybridClock initialized: now 1786032104925479 us; error 39 us; skew 500 ppm
Aug 06 16:01:44 dist-test-slave-tm1g krb5kdc[7553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032104, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:44.932439 7567 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:01:44.933315 7567 webserver.cc:460] Webserver started at http://127.25.31.190:42093/ using document root <none> and password file <none>
I20260806 16:01:44.933898 7567 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:44.933992 7567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:44.934206 7567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:01:44.936245 7567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "34726477f969480d9cdbd4d4d130722d"
format_stamp: "Formatted at 2026-08-06 16:01:44 on dist-test-slave-tm1g"
server_key: "4f97f9b394d7ff6e02324198e09ca6ac"
server_key_iv: "3e94a155dd0c35ce33b3097ae6b9ffe6"
server_key_version: "encryptionkey@0"
I20260806 16:01:44.936801 7567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "34726477f969480d9cdbd4d4d130722d"
format_stamp: "Formatted at 2026-08-06 16:01:44 on dist-test-slave-tm1g"
server_key: "4f97f9b394d7ff6e02324198e09ca6ac"
server_key_iv: "3e94a155dd0c35ce33b3097ae6b9ffe6"
server_key_version: "encryptionkey@0"
I20260806 16:01:44.940579 7567 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260806 16:01:44.943109 7582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:44.944358 7567 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 16:01:44.944535 7567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "34726477f969480d9cdbd4d4d130722d"
format_stamp: "Formatted at 2026-08-06 16:01:44 on dist-test-slave-tm1g"
server_key: "4f97f9b394d7ff6e02324198e09ca6ac"
server_key_iv: "3e94a155dd0c35ce33b3097ae6b9ffe6"
server_key_version: "encryptionkey@0"
I20260806 16:01:44.944658 7567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:44.961474 7567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:44.964493 7567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:44.964697 7567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:44.971565 7567 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:42995
I20260806 16:01:44.971837 7635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:42995 every 8 connection(s)
I20260806 16:01:44.972532 7567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:01:44.975266 7636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:01:44.979611 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 7567
I20260806 16:01:44.979756 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:01:44.980038 25726 external_mini_cluster.cc:1428] Setting key 65bdd399befdd54428186bb2cab68c86
I20260806 16:01:44.980703 7636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: Bootstrap starting.
I20260806 16:01:44.983300 7636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: Neither blocks nor log segments found. Creating new log.
I20260806 16:01:44.984157 7636 log.cc:824] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:44.986268 7636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: No bootstrap required, opened a new log
Aug 06 16:01:44 dist-test-slave-tm1g krb5kdc[7553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032104, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:01:44.990551 7636 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } }
I20260806 16:01:44.990849 7636 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:01:44.990969 7636 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34726477f969480d9cdbd4d4d130722d, State: Initialized, Role: FOLLOWER
I20260806 16:01:44.991482 7636 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } }
I20260806 16:01:44.991652 7636 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:01:44.991721 7636 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:01:44.991818 7636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:01:44.992916 7636 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } }
I20260806 16:01:44.993309 7636 leader_election.cc:302] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34726477f969480d9cdbd4d4d130722d; no voters:
I20260806 16:01:44.993633 7636 leader_election.cc:288] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:01:44.993743 7642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:01:44.994035 7642 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 1 LEADER]: Becoming Leader. State: Replica: 34726477f969480d9cdbd4d4d130722d, State: Running, Role: LEADER
I20260806 16:01:44.994427 7642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } }
I20260806 16:01:44.994895 7636 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:01:44.995666 7642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34726477f969480d9cdbd4d4d130722d. Latest consensus state: current_term: 1 leader_uuid: "34726477f969480d9cdbd4d4d130722d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } } }
I20260806 16:01:44.995831 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:44.981449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51472 (local address 127.25.31.190:42995)
0806 16:01:44.981883 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:44.981902 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:44.981944 (+ 42us) server_negotiation.cc:407] Connection header received
0806 16:01:44.982772 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:44.982793 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:44.983188 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:44.983572 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:44.984266 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:44.985717 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:44.986689 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:44.987347 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:44.987518 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:44.990141 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:44.990147 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:44.990162 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:44.990192 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:44.992294 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:44.992714 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:44.992722 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:44.992728 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:44.992878 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:44.993195 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:44.993199 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:44.993200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:44.993723 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:44.993927 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:44.994584 (+ 657us) server_negotiation.cc:299] Negotiation successful
0806 16:01:44.994887 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":125,"threads_started":1}
I20260806 16:01:44.995836 7642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [sys.catalog]: This master's current role is: LEADER
I20260806 16:01:44.996129 7644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34726477f969480d9cdbd4d4d130722d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } } }
I20260806 16:01:44.996233 7644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [sys.catalog]: This master's current role is: LEADER
I20260806 16:01:44.996539 7651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:01:44.999687 7651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:01:45.005618 7651 catalog_manager.cc:1347] Generated new cluster ID: e00d6c0465a0420f91e5fcf1f67e9005
I20260806 16:01:45.005707 7651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:01:45.021801 7651 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:01:45.022753 7651 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:01:45.032588 7651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: Generated new TSK 0
I20260806 16:01:45.033349 7651 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 16:01:45.061527 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42995
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:01:45.172192 7663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:45.172487 7663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:45.172578 7663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:45.176558 7663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:45.176653 7663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:45.176749 7663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:01:45.181851 7663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:45.182929 7663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:45.183956 7663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:45.194288 7671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:45.194257 7668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:45.194278 7669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:45.194527 7663 server_base.cc:1034] running on GCE node
I20260806 16:01:45.195181 7663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:45.195837 7663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:45.196995 7663 hybrid_clock.cc:648] HybridClock initialized: now 1786032105196980 us; error 27 us; skew 500 ppm
Aug 06 16:01:45 dist-test-slave-tm1g krb5kdc[7553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032105, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:45.199851 7663 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 16:01:45.200663 7663 webserver.cc:460] Webserver started at http://127.25.31.129:39275/ using document root <none> and password file <none>
I20260806 16:01:45.201179 7663 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:45.201236 7663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:45.201395 7663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:01:45.203409 7663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "de2d44e8a19940b3a78c44e026d3f0e4"
format_stamp: "Formatted at 2026-08-06 16:01:45 on dist-test-slave-tm1g"
server_key: "116a2e3259c5d17a5a040f54e369939c"
server_key_iv: "23ee6936d4cd0ed0afd98950ab35f3a8"
server_key_version: "encryptionkey@0"
I20260806 16:01:45.203922 7663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "de2d44e8a19940b3a78c44e026d3f0e4"
format_stamp: "Formatted at 2026-08-06 16:01:45 on dist-test-slave-tm1g"
server_key: "116a2e3259c5d17a5a040f54e369939c"
server_key_iv: "23ee6936d4cd0ed0afd98950ab35f3a8"
server_key_version: "encryptionkey@0"
I20260806 16:01:45.207456 7663 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:01:45.209729 7678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:45.210963 7663 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:01:45.211090 7663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "de2d44e8a19940b3a78c44e026d3f0e4"
format_stamp: "Formatted at 2026-08-06 16:01:45 on dist-test-slave-tm1g"
server_key: "116a2e3259c5d17a5a040f54e369939c"
server_key_iv: "23ee6936d4cd0ed0afd98950ab35f3a8"
server_key_version: "encryptionkey@0"
I20260806 16:01:45.211193 7663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:45.222105 7663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:45.224756 7663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:45.224944 7663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:45.225514 7663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:01:45.226280 7663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:01:45.226387 7663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:45.226473 7663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:01:45.226505 7663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:45.235540 7663 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:43861
I20260806 16:01:45.235576 7794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:43861 every 8 connection(s)
I20260806 16:01:45.236370 7663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 16:01:45.237818 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 7663
I20260806 16:01:45.237955 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 16:01:45.238300 25726 external_mini_cluster.cc:1428] Setting key 3b40041873effb50702e257ec943b9b6
Aug 06 16:01:45 dist-test-slave-tm1g krb5kdc[7553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032105, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
I20260806 16:01:45.251540 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.238188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:39925 (local address 127.25.31.190:42995)
0806 16:01:45.238362 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:45.238369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:45.239306 (+ 937us) server_negotiation.cc:407] Connection header received
0806 16:01:45.241141 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:45.241143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:45.241187 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:45.241255 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:45.242711 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.243460 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.244645 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.244885 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.244995 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.247468 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:45.247472 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:45.247474 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:45.247506 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:45.249327 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:45.249873 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:45.249876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:45.249878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:45.249963 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:45.250328 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:45.250331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:45.250333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:45.250556 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:45.250662 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:45.251226 (+ 564us) server_negotiation.cc:299] Negotiation successful
0806 16:01:45.251351 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 16:01:45.252264 7797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.238624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:39925)
0806 16:01:45.239140 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:45.239198 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:45.240920 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:45.241450 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:45.241462 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:45.242009 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:45.242562 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.242577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.243576 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.243579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.244506 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.244515 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.245037 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.245041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.245214 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.245842 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:45.245858 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:45.247305 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:45.249482 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:45.249487 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:45.249501 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:45.249760 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:45.250058 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:45.250061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:45.250063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:45.250202 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:45.250706 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:45.250712 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:45.250949 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 16:01:45.251166 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 16:01:45.251485 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":142,"threads_started":1}
I20260806 16:01:45.253490 7795 heartbeater.cc:347] Connected to a master server at 127.25.31.190:42995
I20260806 16:01:45.253753 7795 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:45.254223 7795 heartbeater.cc:511] Master 127.25.31.190:42995 requested a full tablet report, sending...
I20260806 16:01:45.255712 7599 ts_manager.cc:194] Registered new tserver with Master: de2d44e8a19940b3a78c44e026d3f0e4 (127.25.31.129:43861)
I20260806 16:01:45.256814 7599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:39925
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 16:01:45.267184 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42995
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:01:45.377523 7800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:45.377756 7800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:45.377797 7800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:45.381810 7800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:45.381866 7800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:45.381922 7800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:01:45.387166 7800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:45.388144 7800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:45.389117 7800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:45.398576 7808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:45.398554 7800 server_base.cc:1034] running on GCE node
W20260806 16:01:45.398598 7806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:45.398540 7805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:45.399168 7800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:45.399816 7800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:45.401006 7800 hybrid_clock.cc:648] HybridClock initialized: now 1786032105400989 us; error 32 us; skew 500 ppm
Aug 06 16:01:45 dist-test-slave-tm1g krb5kdc[7553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032105, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:45.403770 7800 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 16:01:45.404560 7800 webserver.cc:460] Webserver started at http://127.25.31.130:33749/ using document root <none> and password file <none>
I20260806 16:01:45.405066 7800 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:45.405123 7800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:45.405280 7800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:01:45.407289 7800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "a0ce19b733cb444fadded86d8096d6ea"
format_stamp: "Formatted at 2026-08-06 16:01:45 on dist-test-slave-tm1g"
server_key: "4ab8b7015fd2738a4fa954bf8838ecfe"
server_key_iv: "88fcfac14de65bc17745e8b0b615cc17"
server_key_version: "encryptionkey@0"
I20260806 16:01:45.407788 7800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "a0ce19b733cb444fadded86d8096d6ea"
format_stamp: "Formatted at 2026-08-06 16:01:45 on dist-test-slave-tm1g"
server_key: "4ab8b7015fd2738a4fa954bf8838ecfe"
server_key_iv: "88fcfac14de65bc17745e8b0b615cc17"
server_key_version: "encryptionkey@0"
I20260806 16:01:45.411235 7800 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:01:45.413522 7815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:45.414754 7800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:01:45.414970 7800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "a0ce19b733cb444fadded86d8096d6ea"
format_stamp: "Formatted at 2026-08-06 16:01:45 on dist-test-slave-tm1g"
server_key: "4ab8b7015fd2738a4fa954bf8838ecfe"
server_key_iv: "88fcfac14de65bc17745e8b0b615cc17"
server_key_version: "encryptionkey@0"
I20260806 16:01:45.415063 7800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:45.424453 7800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:45.427134 7800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:45.427320 7800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:45.427861 7800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:01:45.428625 7800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:01:45.428671 7800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:45.428740 7800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:01:45.428779 7800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:45.437286 7800 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:34557
I20260806 16:01:45.437328 7930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:34557 every 8 connection(s)
I20260806 16:01:45.438141 7800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:01:45.443224 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 7800
I20260806 16:01:45.443337 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 16:01:45.443642 25726 external_mini_cluster.cc:1428] Setting key 60929d2b75f859a065837e95a212c6d4
Aug 06 16:01:45 dist-test-slave-tm1g krb5kdc[7553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032105, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:01:45.452350 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.439813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:46085 (local address 127.25.31.190:42995)
0806 16:01:45.439981 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:45.439986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:45.440730 (+ 744us) server_negotiation.cc:407] Connection header received
0806 16:01:45.442009 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:45.442012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:45.442059 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:45.442135 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:45.443498 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.444086 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.445355 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.445741 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.445870 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.448596 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:45.448602 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:45.448610 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:45.448634 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:45.450138 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:45.450823 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:45.450827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:45.450829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:45.450925 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:45.451269 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:45.451272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:45.451273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:45.451457 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:45.451561 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:45.452047 (+ 486us) server_negotiation.cc:299] Negotiation successful
0806 16:01:45.452176 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 16:01:45.453086 7933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.440161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:46085)
0806 16:01:45.440582 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:45.440623 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:45.441814 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:45.442289 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:45.442295 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:45.442693 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:45.443289 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.443311 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.444220 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.444223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.445218 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.445227 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.445889 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.445892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.446032 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.446688 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:45.446717 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:45.448439 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:45.450290 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:45.450296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:45.450335 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:45.450721 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:45.451020 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:45.451023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:45.451025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:45.451166 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:45.451573 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:45.451581 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:45.451814 (+ 233us) client_negotiation.cc:769] Sending connection context
0806 16:01:45.451989 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 16:01:45.452297 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
I20260806 16:01:45.454248 7931 heartbeater.cc:347] Connected to a master server at 127.25.31.190:42995
I20260806 16:01:45.454512 7931 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:45.454989 7931 heartbeater.cc:511] Master 127.25.31.190:42995 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
I20260806 16:01:45.456027 7599 ts_manager.cc:194] Registered new tserver with Master: a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557)
I20260806 16:01:45.456594 7599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:46085
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 16:01:45.471055 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42995
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:01:45.580295 7936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:45.580547 7936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:45.580592 7936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:45.584589 7936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:45.584645 7936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:45.584731 7936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:01:45.589830 7936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:45.590914 7936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:45.591899 7936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:45.601613 7942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:45.601626 7944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:45.601629 7941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:45.601806 7936 server_base.cc:1034] running on GCE node
I20260806 16:01:45.602195 7936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:45.602871 7936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:45.604055 7936 hybrid_clock.cc:648] HybridClock initialized: now 1786032105604026 us; error 41 us; skew 500 ppm
Aug 06 16:01:45 dist-test-slave-tm1g krb5kdc[7553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032105, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:45.606709 7936 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 16:01:45.607520 7936 webserver.cc:460] Webserver started at http://127.25.31.131:35433/ using document root <none> and password file <none>
I20260806 16:01:45.608006 7936 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:45.608062 7936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:45.608220 7936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:01:45.610127 7936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "3ca9009352e64d1ab4a671e0c6cef7f1"
format_stamp: "Formatted at 2026-08-06 16:01:45 on dist-test-slave-tm1g"
server_key: "223c7f5951332516670398b34caf134c"
server_key_iv: "4765abb0c3a2c73a4de9074576071b3b"
server_key_version: "encryptionkey@0"
I20260806 16:01:45.610690 7936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "3ca9009352e64d1ab4a671e0c6cef7f1"
format_stamp: "Formatted at 2026-08-06 16:01:45 on dist-test-slave-tm1g"
server_key: "223c7f5951332516670398b34caf134c"
server_key_iv: "4765abb0c3a2c73a4de9074576071b3b"
server_key_version: "encryptionkey@0"
I20260806 16:01:45.614122 7936 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:01:45.616541 7952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:45.617815 7936 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:01:45.617923 7936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "3ca9009352e64d1ab4a671e0c6cef7f1"
format_stamp: "Formatted at 2026-08-06 16:01:45 on dist-test-slave-tm1g"
server_key: "223c7f5951332516670398b34caf134c"
server_key_iv: "4765abb0c3a2c73a4de9074576071b3b"
server_key_version: "encryptionkey@0"
I20260806 16:01:45.618009 7936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:45.631556 7936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:45.634415 7936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:45.634573 7936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:45.635113 7936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:01:45.635911 7936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:01:45.635959 7936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:45.636001 7936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:01:45.636024 7936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:45.645445 7936 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:36715
I20260806 16:01:45.645478 8066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:36715 every 8 connection(s)
I20260806 16:01:45.646430 7936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:01:45.646627 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 7936
I20260806 16:01:45.646744 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 16:01:45.647089 25726 external_mini_cluster.cc:1428] Setting key 081655737b190f3c4d29b29966853966
Aug 06 16:01:45 dist-test-slave-tm1g krb5kdc[7553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032105, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:01:45.659488 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.648363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:35241 (local address 127.25.31.190:42995)
0806 16:01:45.648518 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:45.648523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:45.649514 (+ 991us) server_negotiation.cc:407] Connection header received
0806 16:01:45.650975 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:45.650978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:45.651025 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:45.651097 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:45.652161 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.652758 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.653863 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.654099 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.654188 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.656535 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:45.656539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:45.656541 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:45.656563 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:45.657763 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:45.658202 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:45.658204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:45.658205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:45.658275 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:45.658623 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:45.658626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:45.658627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:45.658792 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:45.658868 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:45.659276 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 16:01:45.659368 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 16:01:45.660269 8069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.648849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:35241)
0806 16:01:45.649366 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:45.649418 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:45.650796 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:45.651221 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:45.651227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:45.651572 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:45.652023 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.652035 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.652874 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.652876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.653734 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.653742 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.654229 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.654232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.654380 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.654993 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:45.655009 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:45.656379 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:45.657865 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:45.657869 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:45.657879 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:45.658117 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:45.658410 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:45.658412 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:45.658413 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:45.658542 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:45.658878 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:45.658883 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:45.659085 (+ 202us) client_negotiation.cc:769] Sending connection context
0806 16:01:45.659229 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 16:01:45.659466 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":132,"threads_started":1}
I20260806 16:01:45.661388 8067 heartbeater.cc:347] Connected to a master server at 127.25.31.190:42995
I20260806 16:01:45.661634 8067 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:45.662087 8067 heartbeater.cc:511] Master 127.25.31.190:42995 requested a full tablet report, sending...
I20260806 16:01:45.663089 7599 ts_manager.cc:194] Registered new tserver with Master: 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715)
I20260806 16:01:45.663633 7599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:35241
I20260806 16:01:45.672760 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:01:45.681936 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.674746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51478 (local address 127.25.31.190:42995)
0806 16:01:45.674893 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:45.674899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:45.675089 (+ 190us) server_negotiation.cc:407] Connection header received
0806 16:01:45.675243 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:45.675247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:45.675289 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:45.675380 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:45.676075 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.676591 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.677426 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.677665 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.677773 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.679089 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:45.679093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:45.679096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:45.679120 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:45.680251 (+ 1131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:45.680672 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:45.680676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:45.680678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:45.680763 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:45.681036 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:45.681039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:45.681041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:45.681252 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:45.681393 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:45.681635 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 16:01:45.681763 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 16:01:45.684572 7599 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:01:45.687636 7599 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:01:45.702076 8081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.696471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:34557 (local address 127.0.0.1:38872)
0806 16:01:45.696679 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:45.696692 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:45.696755 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:45.697879 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:45.697884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:45.697904 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:01:45.698017 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.698023 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.699370 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.699374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.700395 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.700416 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.701483 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.701486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.701612 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.701669 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 16:01:45.701777 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 16:01:45.701899 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":44,"thread_start_us":101,"threads_started":1}
I20260806 16:01:45.702093 8083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.696985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38872 (local address 127.25.31.130:34557)
0806 16:01:45.697353 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:45.697365 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:45.697385 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:01:45.697455 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:45.697460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:45.697638 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:45.697771 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:01:45.698161 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.699246 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.700511 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.701350 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.701474 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.701615 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:45.701825 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 16:01:45.701940 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":131,"threads_started":1}
I20260806 16:01:45.702375 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.695733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:36715 (local address 127.0.0.1:37838)
0806 16:01:45.696227 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:45.696282 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:45.696403 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:45.697366 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:45.697373 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:45.697401 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:01:45.697546 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.697553 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.699167 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.699170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.700420 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.700431 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.701953 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.701957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.702052 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.702075 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 16:01:45.702130 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 16:01:45.702210 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":33,"thread_start_us":106,"threads_started":1}
I20260806 16:01:45.702709 8082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.696682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56730 (local address 127.25.31.129:43861)
0806 16:01:45.697018 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:45.697032 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:45.697054 (+ 22us) server_negotiation.cc:407] Connection header received
0806 16:01:45.697139 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:45.697145 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:45.697337 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:45.697503 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:01:45.697927 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.698790 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.700961 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.701564 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.701667 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.701806 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:45.702454 (+ 648us) server_negotiation.cc:299] Negotiation successful
0806 16:01:45.702567 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":119,"threads_started":1}
I20260806 16:01:45.702979 8079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.695951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37838 (local address 127.25.31.131:36715)
0806 16:01:45.696858 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:45.696871 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:45.696895 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:01:45.696974 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:45.696981 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:45.697174 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:45.697705 (+ 531us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:01:45.697865 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.699034 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.700991 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.701804 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.702542 (+ 738us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.702694 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:45.702744 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 16:01:45.702828 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":162,"threads_started":1}
I20260806 16:01:45.703014 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.696115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:43861 (local address 127.0.0.1:56730)
0806 16:01:45.696645 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:45.696659 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:45.696745 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:45.697681 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:45.697684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:45.697703 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:01:45.697796 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.697805 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.698921 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.698926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.700841 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.700850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.701865 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.701869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.701961 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.701983 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 16:01:45.702050 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 16:01:45.702123 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":388,"thread_start_us":100,"threads_started":1}
I20260806 16:01:45.705382 8001 tablet_service.cc:1467] Processing CreateTablet for tablet 76d46e53cd5542afb0d2ff4a585b8adb (DEFAULT_TABLE table=test-table [id=b781d6d292264d0d9b25779b9586c674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:01:45.705415 7864 tablet_service.cc:1467] Processing CreateTablet for tablet 76d46e53cd5542afb0d2ff4a585b8adb (DEFAULT_TABLE table=test-table [id=b781d6d292264d0d9b25779b9586c674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:01:45.705766 7727 tablet_service.cc:1467] Processing CreateTablet for tablet 76d46e53cd5542afb0d2ff4a585b8adb (DEFAULT_TABLE table=test-table [id=b781d6d292264d0d9b25779b9586c674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:01:45.706475 8001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76d46e53cd5542afb0d2ff4a585b8adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:01:45.706480 7864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76d46e53cd5542afb0d2ff4a585b8adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:01:45.706684 7727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76d46e53cd5542afb0d2ff4a585b8adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:01:45.712805 8084 tablet_bootstrap.cc:492] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1: Bootstrap starting.
I20260806 16:01:45.713996 8085 tablet_bootstrap.cc:492] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: Bootstrap starting.
I20260806 16:01:45.714670 8086 tablet_bootstrap.cc:492] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: Bootstrap starting.
I20260806 16:01:45.715267 8084 tablet_bootstrap.cc:654] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1: Neither blocks nor log segments found. Creating new log.
I20260806 16:01:45.716156 8085 tablet_bootstrap.cc:654] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: Neither blocks nor log segments found. Creating new log.
I20260806 16:01:45.716481 8084 log.cc:824] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:45.717191 8086 tablet_bootstrap.cc:654] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: Neither blocks nor log segments found. Creating new log.
I20260806 16:01:45.717527 8085 log.cc:824] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:45.718047 8086 log.cc:824] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:45.718914 8084 tablet_bootstrap.cc:492] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1: No bootstrap required, opened a new log
I20260806 16:01:45.719099 8084 ts_tablet_manager.cc:1397] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 16:01:45.719553 8085 tablet_bootstrap.cc:492] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: No bootstrap required, opened a new log
I20260806 16:01:45.719731 8085 ts_tablet_manager.cc:1397] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 16:01:45.719961 8086 tablet_bootstrap.cc:492] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: No bootstrap required, opened a new log
I20260806 16:01:45.720127 8086 ts_tablet_manager.cc:1397] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 16:01:45.722211 8084 raft_consensus.cc:348] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:45.722536 8084 raft_consensus.cc:374] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:01:45.722615 8084 raft_consensus.cc:729] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ca9009352e64d1ab4a671e0c6cef7f1, State: Initialized, Role: FOLLOWER
I20260806 16:01:45.723078 8084 consensus_queue.cc:260] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:45.723673 8067 heartbeater.cc:503] Master 127.25.31.190:42995 was elected leader, sending a full tablet report...
I20260806 16:01:45.723670 8085 raft_consensus.cc:348] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:45.723764 8086 raft_consensus.cc:348] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:45.723927 8085 raft_consensus.cc:374] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:01:45.723986 8085 raft_consensus.cc:729] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2d44e8a19940b3a78c44e026d3f0e4, State: Initialized, Role: FOLLOWER
I20260806 16:01:45.724030 8086 raft_consensus.cc:374] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:01:45.724100 8086 raft_consensus.cc:729] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ce19b733cb444fadded86d8096d6ea, State: Initialized, Role: FOLLOWER
I20260806 16:01:45.724485 8085 consensus_queue.cc:260] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:45.724520 8086 consensus_queue.cc:260] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:45.725483 8085 ts_tablet_manager.cc:1428] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 16:01:45.725481 8086 ts_tablet_manager.cc:1428] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 16:01:45.725965 7931 heartbeater.cc:503] Master 127.25.31.190:42995 was elected leader, sending a full tablet report...
I20260806 16:01:45.726642 7795 heartbeater.cc:503] Master 127.25.31.190:42995 was elected leader, sending a full tablet report...
I20260806 16:01:45.726854 8084 ts_tablet_manager.cc:1428] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1: Time spent starting tablet: real 0.008s user 0.002s sys 0.003s
W20260806 16:01:45.737452 7796 tablet.cc:2369] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:01:45.794793 8092 raft_consensus.cc:482] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:01:45.795019 8092 raft_consensus.cc:504] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:45.796312 8092 leader_election.cc:288] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715), a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557)
I20260806 16:01:45.800973 8079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.796676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:39589 (local address 127.25.31.131:36715)
0806 16:01:45.796814 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:45.796817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:45.796920 (+ 103us) server_negotiation.cc:407] Connection header received
0806 16:01:45.797097 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:45.797100 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:45.797178 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:45.797263 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:01:45.797639 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.798288 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.799793 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.800397 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.800528 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.800600 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:45.800756 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 16:01:45.800846 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 16:01:45.800997 8094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.797004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:34557 (local address 127.25.31.129:54197)
0806 16:01:45.797343 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:45.797358 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:45.797454 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:45.797712 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:45.797718 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:45.797737 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:01:45.797840 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.797846 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.798701 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.798705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.799816 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.799830 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.800619 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.800622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.800738 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.800756 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 16:01:45.800819 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 16:01:45.800872 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":109,"threads_started":1}
I20260806 16:01:45.801043 8083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.797189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:54197 (local address 127.25.31.130:34557)
0806 16:01:45.797328 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:45.797331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:45.797421 (+ 90us) server_negotiation.cc:407] Connection header received
0806 16:01:45.797541 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:45.797545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:45.797615 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:45.797692 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:01:45.797967 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.798601 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.799925 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.800521 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.800628 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.800671 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:45.800847 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 16:01:45.800919 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 16:01:45.800999 8093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.796586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:36715 (local address 127.25.31.129:39589)
0806 16:01:45.796853 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:45.796868 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:45.796975 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:45.797379 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:45.797382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:45.797400 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:01:45.797514 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.797518 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.798447 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.798454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.799669 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:45.799679 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.800518 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.800521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:45.800620 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.800649 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 16:01:45.800726 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 16:01:45.800808 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":111,"threads_started":1}
I20260806 16:01:45.801775 8021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76d46e53cd5542afb0d2ff4a585b8adb" candidate_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" is_pre_election: true
I20260806 16:01:45.801805 7885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76d46e53cd5542afb0d2ff4a585b8adb" candidate_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ce19b733cb444fadded86d8096d6ea" is_pre_election: true
I20260806 16:01:45.802088 8021 raft_consensus.cc:2454] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de2d44e8a19940b3a78c44e026d3f0e4 in term 0.
I20260806 16:01:45.802098 7885 raft_consensus.cc:2454] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de2d44e8a19940b3a78c44e026d3f0e4 in term 0.
I20260806 16:01:45.802543 7681 leader_election.cc:302] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0ce19b733cb444fadded86d8096d6ea, de2d44e8a19940b3a78c44e026d3f0e4; no voters:
I20260806 16:01:45.802781 8092 raft_consensus.cc:2781] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:01:45.802870 8092 raft_consensus.cc:482] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:01:45.802934 8092 raft_consensus.cc:3037] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:01:45.803948 8092 raft_consensus.cc:504] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:45.804270 8092 leader_election.cc:288] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 1 election: Requested vote from peers 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715), a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557)
I20260806 16:01:45.804612 8021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76d46e53cd5542afb0d2ff4a585b8adb" candidate_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1"
I20260806 16:01:45.804656 7885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76d46e53cd5542afb0d2ff4a585b8adb" candidate_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ce19b733cb444fadded86d8096d6ea"
I20260806 16:01:45.804740 8021 raft_consensus.cc:3037] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:01:45.804755 7885 raft_consensus.cc:3037] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:01:45.806032 7885 raft_consensus.cc:2454] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de2d44e8a19940b3a78c44e026d3f0e4 in term 1.
I20260806 16:01:45.806032 8021 raft_consensus.cc:2454] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de2d44e8a19940b3a78c44e026d3f0e4 in term 1.
I20260806 16:01:45.806355 7681 leader_election.cc:302] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ca9009352e64d1ab4a671e0c6cef7f1, de2d44e8a19940b3a78c44e026d3f0e4; no voters:
I20260806 16:01:45.806531 8092 raft_consensus.cc:2781] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:01:45.806737 8092 raft_consensus.cc:686] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 LEADER]: Becoming Leader. State: Replica: de2d44e8a19940b3a78c44e026d3f0e4, State: Running, Role: LEADER
I20260806 16:01:45.807080 8092 consensus_queue.cc:237] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:45.810122 7599 catalog_manager.cc:5495] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to de2d44e8a19940b3a78c44e026d3f0e4 (127.25.31.129). New cstate: current_term: 1 leader_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } health_report { overall_health: HEALTHY } } }
I20260806 16:01:45.822805 8082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.819227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56736 (local address 127.25.31.129:43861)
0806 16:01:45.819360 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:45.819364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:45.819378 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:01:45.819495 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:45.819497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:45.819583 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:45.819692 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:01:45.819905 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.820434 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.821298 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:45.821551 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:45.821628 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:45.821905 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:01:45.822377 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:01:45.822471 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:45.822587 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 16:01:45.822637 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 16:01:45.828320 8021 raft_consensus.cc:1261] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 1 FOLLOWER]: Refusing update from remote peer de2d44e8a19940b3a78c44e026d3f0e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:01:45.828347 7885 raft_consensus.cc:1261] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea [term 1 FOLLOWER]: Refusing update from remote peer de2d44e8a19940b3a78c44e026d3f0e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:01:45.829059 8095 consensus_queue.cc:1035] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:01:45.829236 8092 consensus_queue.cc:1035] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:01:45.837518 8100 mvcc.cc:205] Tried to move back new op lower bound from 7315587505466073088 to 7315587505388539904. Current Snapshot: MvccSnapshot[applied={T|T < 7315587505466073088}]
I20260806 16:01:45.844606 7598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:01:45.844739 7598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:01:45.847105 7598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: Sending DeleteTablet for 3 replicas of tablet 76d46e53cd5542afb0d2ff4a585b8adb
I20260806 16:01:45.847872 7727 tablet_service.cc:1514] Processing DeleteTablet for tablet 76d46e53cd5542afb0d2ff4a585b8adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:01:45 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:56730
I20260806 16:01:45.847874 8001 tablet_service.cc:1514] Processing DeleteTablet for tablet 76d46e53cd5542afb0d2ff4a585b8adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:01:45 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:37838
I20260806 16:01:45.847924 7864 tablet_service.cc:1514] Processing DeleteTablet for tablet 76d46e53cd5542afb0d2ff4a585b8adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:01:45 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:38872
I20260806 16:01:45.848340 8108 tablet_replica.cc:330] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: stopping tablet replica
I20260806 16:01:45.848649 8108 raft_consensus.cc:2229] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:01:45.848712 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 7663
I20260806 16:01:45.854532 8109 tablet_replica.cc:330] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: stopping tablet replica
I20260806 16:01:45.854552 8110 tablet_replica.cc:330] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1: stopping tablet replica
I20260806 16:01:45.854826 8109 raft_consensus.cc:2229] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:01:45.854867 8110 raft_consensus.cc:2229] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:01:45.855142 8109 raft_consensus.cc:2258] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:01:45.855211 8110 raft_consensus.cc:2258] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:01:45.856654 8109 ts_tablet_manager.cc:1905] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:01:45.857034 8110 ts_tablet_manager.cc:1905] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 16:01:45.857187 7586 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:43861 (error 108)
I20260806 16:01:45.857442 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 7800
I20260806 16:01:45.860317 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.860043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:43861 (local address 127.0.0.1:56742)
0806 16:01:45.860171 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:45.860243 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:43861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 16:01:45.860517 7586 catalog_manager.cc:4584] TS de2d44e8a19940b3a78c44e026d3f0e4 (127.25.31.129:43861): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 76d46e53cd5542afb0d2ff4a585b8adb: Network error: Client connection negotiation failed: client connection to 127.25.31.129:43861: connect: Connection refused (error 111)
I20260806 16:01:45.861766 8110 ts_tablet_manager.cc:1918] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:01:45.861883 8110 log.cc:1196] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/76d46e53cd5542afb0d2ff4a585b8adb
I20260806 16:01:45.862264 8110 ts_tablet_manager.cc:1939] T 76d46e53cd5542afb0d2ff4a585b8adb P 3ca9009352e64d1ab4a671e0c6cef7f1: Deleting consensus metadata
I20260806 16:01:45.863636 7585 catalog_manager.cc:4841] TS 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715): tablet 76d46e53cd5542afb0d2ff4a585b8adb (table test-table [id=b781d6d292264d0d9b25779b9586c674]) successfully deleted
I20260806 16:01:45.865821 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 7936
I20260806 16:01:45.866156 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:45.865865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:34557 (local address 127.0.0.1:38880)
0806 16:01:45.866000 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:45.866064 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.130:34557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 16:01:45.872213 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 7567
I20260806 16:01:45.878822 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42995
--webserver_interface=127.25.31.190
--webserver_port=42093
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:42995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:01:45.993935 8112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:45.994220 8112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:45.994333 8112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:45.998418 8112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:01:45.998490 8112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:45.998528 8112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:45.998590 8112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:01:45.998616 8112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:01:46.003827 8112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:42995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=42093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:46.004882 8112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:46.005925 8112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:46.020298 8118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:46.020305 8119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:46.020453 8112 server_base.cc:1034] running on GCE node
W20260806 16:01:46.020633 8121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:46.021257 8112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:46.022449 8112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:46.023631 8112 hybrid_clock.cc:648] HybridClock initialized: now 1786032106023606 us; error 45 us; skew 500 ppm
Aug 06 16:01:46 dist-test-slave-tm1g krb5kdc[7553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032106, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:46.026517 8112 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:01:46.027434 8112 webserver.cc:460] Webserver started at http://127.25.31.190:42093/ using document root <none> and password file <none>
I20260806 16:01:46.028023 8112 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:46.028122 8112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:46.032106 8112 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 16:01:46.034401 8128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:46.039248 8112 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260806 16:01:46.039417 8112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "34726477f969480d9cdbd4d4d130722d"
format_stamp: "Formatted at 2026-08-06 16:01:44 on dist-test-slave-tm1g"
server_key: "4f97f9b394d7ff6e02324198e09ca6ac"
server_key_iv: "3e94a155dd0c35ce33b3097ae6b9ffe6"
server_key_version: "encryptionkey@0"
I20260806 16:01:46.040221 8112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:46.066474 8112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:46.069344 8112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:46.069547 8112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:46.077061 8112 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:42995
I20260806 16:01:46.077440 8180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:42995 every 8 connection(s)
I20260806 16:01:46.078233 8112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:01:46.082659 8181 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:01:46.084270 8181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: Bootstrap starting.
I20260806 16:01:46.087590 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 8112
I20260806 16:01:46.088079 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:43861
--local_ip_for_outbound_sockets=127.25.31.129
--tserver_master_addrs=127.25.31.190:42995
--webserver_port=39275
--webserver_interface=127.25.31.129
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:01:46.106745 8181 log.cc:824] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:46.117832 8181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:01:46.118247 8181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: Bootstrap complete.
I20260806 16:01:46.123773 8181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } }
I20260806 16:01:46.124303 8181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34726477f969480d9cdbd4d4d130722d, State: Initialized, Role: FOLLOWER
I20260806 16:01:46.124838 8181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } }
I20260806 16:01:46.124981 8181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:01:46.125058 8181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:01:46.125154 8181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 1 FOLLOWER]: Advancing to term 2
I20260806 16:01:46.127604 8181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } }
I20260806 16:01:46.127971 8181 leader_election.cc:302] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34726477f969480d9cdbd4d4d130722d; no voters:
I20260806 16:01:46.128306 8181 leader_election.cc:288] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 16:01:46.128438 8185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 2 FOLLOWER]: Leader election won for term 2
I20260806 16:01:46.128727 8185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 2 LEADER]: Becoming Leader. State: Replica: 34726477f969480d9cdbd4d4d130722d, State: Running, Role: LEADER
I20260806 16:01:46.129086 8185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } }
I20260806 16:01:46.129515 8181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:01:46.130519 8186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "34726477f969480d9cdbd4d4d130722d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } } }
I20260806 16:01:46.130666 8186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [sys.catalog]: This master's current role is: LEADER
I20260806 16:01:46.131114 8187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34726477f969480d9cdbd4d4d130722d. Latest consensus state: current_term: 2 leader_uuid: "34726477f969480d9cdbd4d4d130722d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } } }
I20260806 16:01:46.131264 8187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [sys.catalog]: This master's current role is: LEADER
I20260806 16:01:46.131611 8195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:01:46.136471 8195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:01:46.137095 8195 catalog_manager.cc:1259] Loaded cluster ID: e00d6c0465a0420f91e5fcf1f67e9005
I20260806 16:01:46.137156 8195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:01:46.138593 8195 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 16:01:46.143487 8195 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 16:01:46.148908 8195 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:01:46.149657 8195 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: Loaded TSK: 0
I20260806 16:01:46.150841 8195 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:01:46.216274 8183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:46.216563 8183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:46.216666 8183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:46.220770 8183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:46.220860 8183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:46.220953 8183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:01:46.226251 8183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:43861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=39275
--tserver_master_addrs=127.25.31.190:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:46.227319 8183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:46.228351 8183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:46.238021 8212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:46.238021 8209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:46.238511 8210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:46.238632 8183 server_base.cc:1034] running on GCE node
I20260806 16:01:46.239032 8183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:46.239675 8183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:46.240857 8183 hybrid_clock.cc:648] HybridClock initialized: now 1786032106240837 us; error 38 us; skew 500 ppm
Aug 06 16:01:46 dist-test-slave-tm1g krb5kdc[7553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032106, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:46.243605 8183 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 16:01:46.244547 8183 webserver.cc:460] Webserver started at http://127.25.31.129:39275/ using document root <none> and password file <none>
I20260806 16:01:46.245146 8183 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:46.245311 8183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:46.249241 8183 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260806 16:01:46.251505 8219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:46.252759 8183 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:01:46.252892 8183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "de2d44e8a19940b3a78c44e026d3f0e4"
format_stamp: "Formatted at 2026-08-06 16:01:45 on dist-test-slave-tm1g"
server_key: "116a2e3259c5d17a5a040f54e369939c"
server_key_iv: "23ee6936d4cd0ed0afd98950ab35f3a8"
server_key_version: "encryptionkey@0"
I20260806 16:01:46.253429 8183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:46.266902 8183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:46.270176 8183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:46.270432 8183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:46.271035 8183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:01:46.272176 8226 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 16:01:46.277328 8183 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 16:01:46.277400 8183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260806 16:01:46.277477 8183 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 16:01:46.279178 8183 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 16:01:46.279247 8183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 16:01:46.279522 8226 tablet_bootstrap.cc:492] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: Bootstrap starting.
I20260806 16:01:46.289764 8183 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:43861
I20260806 16:01:46.289876 8337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:43861 every 8 connection(s)
I20260806 16:01:46.290746 8183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 16:01:46.295959 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 8183
I20260806 16:01:46.296388 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:34557
--local_ip_for_outbound_sockets=127.25.31.130
--tserver_master_addrs=127.25.31.190:42995
--webserver_port=33749
--webserver_interface=127.25.31.130
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:01:46.299642 8226 log.cc:824] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:01:46 dist-test-slave-tm1g krb5kdc[7553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032106, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:01:46.308569 8226 tablet_bootstrap.cc:492] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:01:46.309113 8226 tablet_bootstrap.cc:492] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: Bootstrap complete.
I20260806 16:01:46.309912 8342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.292646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:42019)
0806 16:01:46.293233 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:46.293292 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:46.294577 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:46.295694 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:46.295709 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:46.296256 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:46.296922 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.296949 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.298695 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.298699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.299597 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.299607 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.300394 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.300398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.300606 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.301257 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:46.301283 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:46.303396 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:46.306039 (+ 2643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:46.306048 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:46.306066 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:46.306490 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:46.306913 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:46.306920 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:46.306924 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:46.307124 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:46.307853 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:46.307861 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:46.308189 (+ 328us) client_negotiation.cc:769] Sending connection context
0806 16:01:46.308404 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 16:01:46.308860 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":145,"threads_started":1}
I20260806 16:01:46.310540 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.292770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:42019 (local address 127.25.31.190:42995)
0806 16:01:46.293237 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:46.293255 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:46.293425 (+ 170us) server_negotiation.cc:407] Connection header received
0806 16:01:46.294983 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:46.294994 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:46.295270 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:46.295534 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:46.297139 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.298526 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.299735 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.300241 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.300424 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.303619 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:46.303625 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:46.303641 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:46.303673 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:46.305898 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:46.306635 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:46.306641 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:46.306646 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:46.306796 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:46.307253 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:46.307256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:46.307258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:46.307706 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:46.307842 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:46.309350 (+ 1508us) server_negotiation.cc:299] Negotiation successful
0806 16:01:46.309663 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":108,"threads_started":1}
I20260806 16:01:46.310739 8226 ts_tablet_manager.cc:1397] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: Time spent bootstrapping tablet: real 0.031s user 0.014s sys 0.016s
I20260806 16:01:46.312961 8338 heartbeater.cc:347] Connected to a master server at 127.25.31.190:42995
I20260806 16:01:46.313190 8338 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:46.313701 8338 heartbeater.cc:511] Master 127.25.31.190:42995 requested a full tablet report, sending...
I20260806 16:01:46.313473 8226 raft_consensus.cc:348] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:46.313958 8226 raft_consensus.cc:729] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2d44e8a19940b3a78c44e026d3f0e4, State: Initialized, Role: FOLLOWER
I20260806 16:01:46.314575 8226 consensus_queue.cc:260] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:46.315661 8226 ts_tablet_manager.cc:1428] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 16:01:46.317654 8145 ts_manager.cc:194] Registered new tserver with Master: de2d44e8a19940b3a78c44e026d3f0e4 (127.25.31.129:43861)
I20260806 16:01:46.321631 8145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:42019
I20260806 16:01:46.323971 8338 heartbeater.cc:503] Master 127.25.31.190:42995 was elected leader, sending a full tablet report...
Aug 06 16:01:46 dist-test-slave-tm1g krb5kdc[7553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032106, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for kudu/127.25.31.129@KRBTEST.COM
I20260806 16:01:46.333571 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.320988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:43861 (local address 127.0.0.1:56754)
0806 16:01:46.321871 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:46.322019 (+ 148us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:46.322173 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:46.322755 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:46.324320 (+ 1565us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:46.324799 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:46.324926 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.324951 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.326391 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.326394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.327018 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.327027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.327760 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.327763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.327859 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.328451 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:46.328463 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:46.329804 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:46.331931 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:46.331940 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:46.331944 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:46.332223 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:46.332525 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:46.332527 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:46.332528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:46.332603 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:46.333085 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:46.333089 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:46.333185 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 16:01:46.333292 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 16:01:46.333444 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":96,"threads_started":1}
I20260806 16:01:46.333714 8347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.321640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56754 (local address 127.25.31.129:43861)
0806 16:01:46.322032 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:46.322046 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:46.322094 (+ 48us) server_negotiation.cc:407] Connection header received
0806 16:01:46.322387 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:46.322392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:46.322608 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:46.322755 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:46.325089 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.326196 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.327138 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.327624 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.327742 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.329954 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:46.329964 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:46.329976 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:46.330003 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:46.331827 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:46.332333 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:46.332339 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:46.332341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:46.332435 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:46.332687 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:46.332690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:46.332692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:46.332978 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:46.333107 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:46.333395 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 16:01:46.333560 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":108,"threads_started":1}
I20260806 16:01:46.334829 8269 tablet_service.cc:1514] Processing DeleteTablet for tablet 76d46e53cd5542afb0d2ff4a585b8adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:01:45 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:56754
I20260806 16:01:46.335790 8348 tablet_replica.cc:330] stopping tablet replica
I20260806 16:01:46.336086 8348 raft_consensus.cc:2229] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:01:46.336325 8348 raft_consensus.cc:2258] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:01:46.337252 8348 ts_tablet_manager.cc:1905] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:01:46.340662 8348 ts_tablet_manager.cc:1918] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:01:46.340754 8348 log.cc:1196] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal/wals/76d46e53cd5542afb0d2ff4a585b8adb
I20260806 16:01:46.341078 8348 ts_tablet_manager.cc:1939] T 76d46e53cd5542afb0d2ff4a585b8adb P de2d44e8a19940b3a78c44e026d3f0e4: Deleting consensus metadata
I20260806 16:01:46.342758 8132 catalog_manager.cc:4841] TS de2d44e8a19940b3a78c44e026d3f0e4 (127.25.31.129:43861): tablet 76d46e53cd5542afb0d2ff4a585b8adb (table test-table [id=b781d6d292264d0d9b25779b9586c674]) successfully deleted
W20260806 16:01:46.416628 8343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:46.416934 8343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:46.417012 8343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:46.420965 8343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:46.421058 8343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:46.421149 8343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:01:46.426231 8343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:34557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=33749
--tserver_master_addrs=127.25.31.190:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:46.427304 8343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:46.428334 8343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:46.438007 8353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:46.438066 8354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:46.438014 8356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:46.438217 8343 server_base.cc:1034] running on GCE node
I20260806 16:01:46.438719 8343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:46.439338 8343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:46.440544 8343 hybrid_clock.cc:648] HybridClock initialized: now 1786032106440508 us; error 55 us; skew 500 ppm
Aug 06 16:01:46 dist-test-slave-tm1g krb5kdc[7553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032106, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:46.443161 8343 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 16:01:46.444031 8343 webserver.cc:460] Webserver started at http://127.25.31.130:33749/ using document root <none> and password file <none>
I20260806 16:01:46.444577 8343 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:46.444712 8343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:46.448505 8343 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260806 16:01:46.450660 8363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:46.451846 8343 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260806 16:01:46.451947 8343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "a0ce19b733cb444fadded86d8096d6ea"
format_stamp: "Formatted at 2026-08-06 16:01:45 on dist-test-slave-tm1g"
server_key: "4ab8b7015fd2738a4fa954bf8838ecfe"
server_key_iv: "88fcfac14de65bc17745e8b0b615cc17"
server_key_version: "encryptionkey@0"
I20260806 16:01:46.452430 8343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:46.467478 8343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:46.470353 8343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:46.470504 8343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:46.471012 8343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:01:46.472007 8370 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 16:01:46.476219 8343 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 16:01:46.476305 8343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260806 16:01:46.476387 8343 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 16:01:46.477988 8343 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 16:01:46.478070 8343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 16:01:46.478288 8370 tablet_bootstrap.cc:492] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: Bootstrap starting.
I20260806 16:01:46.483888 8370 log.cc:824] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:46.488305 8343 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:34557
I20260806 16:01:46.488360 8482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:34557 every 8 connection(s)
I20260806 16:01:46.489120 8343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:01:46.493008 8370 tablet_bootstrap.cc:492] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:01:46.493458 8370 tablet_bootstrap.cc:492] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: Bootstrap complete.
I20260806 16:01:46.493806 8370 ts_tablet_manager.cc:1397] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.004s
I20260806 16:01:46.494613 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 8343
I20260806 16:01:46.495054 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:36715
--local_ip_for_outbound_sockets=127.25.31.131
--tserver_master_addrs=127.25.31.190:42995
--webserver_port=35433
--webserver_interface=127.25.31.131
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:01:46.496775 8370 raft_consensus.cc:348] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:46.497305 8370 raft_consensus.cc:729] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ce19b733cb444fadded86d8096d6ea, State: Initialized, Role: FOLLOWER
I20260806 16:01:46.497815 8370 consensus_queue.cc:260] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:46.498917 8370 ts_tablet_manager.cc:1428] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
Aug 06 16:01:46 dist-test-slave-tm1g krb5kdc[7553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032106, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:01:46.504379 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.491029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:47919 (local address 127.25.31.190:42995)
0806 16:01:46.491192 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:46.491198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:46.491819 (+ 621us) server_negotiation.cc:407] Connection header received
0806 16:01:46.492919 (+ 1100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:46.492922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:46.492983 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:46.493068 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:46.494601 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.495428 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.497049 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.497466 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.497597 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.500696 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:46.500700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:46.500703 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:46.500738 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:46.502238 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:46.502845 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:46.502849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:46.502851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:46.502942 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:46.503285 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:46.503290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:46.503292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:46.503483 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:46.503592 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:46.504060 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 16:01:46.504199 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 16:01:46.505159 8485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.491141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:47919)
0806 16:01:46.491647 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:46.491701 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:46.492692 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:46.493254 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:46.493264 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:46.493752 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:46.494428 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.494449 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.495613 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.495618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.496909 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.496920 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.497644 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.497648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.497847 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.499040 (+ 1193us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:46.499059 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:46.500516 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:46.502423 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:46.502428 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:46.502454 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:46.502734 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:46.503040 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:46.503043 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:46.503045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:46.503187 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:46.503590 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:46.503597 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:46.503831 (+ 234us) client_negotiation.cc:769] Sending connection context
0806 16:01:46.504016 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 16:01:46.504332 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":127,"threads_started":1}
I20260806 16:01:46.506331 8483 heartbeater.cc:347] Connected to a master server at 127.25.31.190:42995
I20260806 16:01:46.506589 8483 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:46.507058 8483 heartbeater.cc:511] Master 127.25.31.190:42995 requested a full tablet report, sending...
I20260806 16:01:46.508651 8145 ts_manager.cc:194] Registered new tserver with Master: a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557)
I20260806 16:01:46.509809 8145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:47919
I20260806 16:01:46.511778 8483 heartbeater.cc:503] Master 127.25.31.190:42995 was elected leader, sending a full tablet report...
Aug 06 16:01:46 dist-test-slave-tm1g krb5kdc[7553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032106, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for kudu/127.25.31.130@KRBTEST.COM
I20260806 16:01:46.518852 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.509536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:34557 (local address 127.0.0.1:38886)
0806 16:01:46.509701 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:46.509716 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:46.509786 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:46.510482 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:46.510485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:46.510661 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:46.510755 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.510762 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.512183 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.512189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.512972 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.512982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.513658 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.513661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.513760 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.514214 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:46.514225 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:46.515469 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:46.517235 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:46.517238 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:46.517241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:46.517442 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:46.517749 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:46.517752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:46.517754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:46.517838 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:46.518277 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:46.518280 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:46.518419 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 16:01:46.518543 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 16:01:46.518685 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 16:01:46.518944 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.509641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38886 (local address 127.25.31.130:34557)
0806 16:01:46.510028 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:46.510039 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:46.510060 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:01:46.510121 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:46.510126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:46.510364 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:46.510491 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:46.510876 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.512079 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.513076 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.513534 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.513645 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.515603 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:46.515613 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:46.515625 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:46.515655 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:46.517120 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:46.517556 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:46.517561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:46.517563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:46.517652 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:46.517928 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:46.517941 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:46.517943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:46.518169 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:46.518311 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:46.518626 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 16:01:46.518777 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":85,"threads_started":1}
I20260806 16:01:46.520215 8414 tablet_service.cc:1514] Processing DeleteTablet for tablet 76d46e53cd5542afb0d2ff4a585b8adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:01:45 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:38886
I20260806 16:01:46.521102 8489 tablet_replica.cc:330] stopping tablet replica
I20260806 16:01:46.521389 8489 raft_consensus.cc:2229] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:01:46.521623 8489 raft_consensus.cc:2258] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:01:46.522589 8489 ts_tablet_manager.cc:1905] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:01:46.526032 8489 ts_tablet_manager.cc:1918] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:01:46.526116 8489 log.cc:1196] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/76d46e53cd5542afb0d2ff4a585b8adb
I20260806 16:01:46.526491 8489 ts_tablet_manager.cc:1939] T 76d46e53cd5542afb0d2ff4a585b8adb P a0ce19b733cb444fadded86d8096d6ea: Deleting consensus metadata
I20260806 16:01:46.527957 8131 catalog_manager.cc:4841] TS a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557): tablet 76d46e53cd5542afb0d2ff4a585b8adb (table test-table [id=b781d6d292264d0d9b25779b9586c674]) successfully deleted
W20260806 16:01:46.609611 8486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:46.609900 8486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:46.609977 8486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:46.614010 8486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:46.614099 8486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:46.614192 8486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:01:46.619325 8486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:36715
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=35433
--tserver_master_addrs=127.25.31.190:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:46.620333 8486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:46.621308 8486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:46.630736 8494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:46.630752 8495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:46.630759 8497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:46.630962 8486 server_base.cc:1034] running on GCE node
I20260806 16:01:46.631377 8486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:46.632051 8486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:46.633208 8486 hybrid_clock.cc:648] HybridClock initialized: now 1786032106633180 us; error 39 us; skew 500 ppm
Aug 06 16:01:46 dist-test-slave-tm1g krb5kdc[7553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032106, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:46.635910 8486 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 16:01:46.636770 8486 webserver.cc:460] Webserver started at http://127.25.31.131:35433/ using document root <none> and password file <none>
I20260806 16:01:46.637301 8486 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:46.637411 8486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:46.641266 8486 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:01:46.643276 8505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:46.644549 8486 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:01:46.644677 8486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "3ca9009352e64d1ab4a671e0c6cef7f1"
format_stamp: "Formatted at 2026-08-06 16:01:45 on dist-test-slave-tm1g"
server_key: "223c7f5951332516670398b34caf134c"
server_key_iv: "4765abb0c3a2c73a4de9074576071b3b"
server_key_version: "encryptionkey@0"
I20260806 16:01:46.645243 8486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:46.654767 8486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:46.657935 8486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:46.658116 8486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:46.658723 8486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:01:46.659507 8486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:01:46.659584 8486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:46.659674 8486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:01:46.659732 8486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:46.668483 8486 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:36715
I20260806 16:01:46.668504 8621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:36715 every 8 connection(s)
I20260806 16:01:46.669430 8486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:01:46.672540 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 8486
Aug 06 16:01:46 dist-test-slave-tm1g krb5kdc[7553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032106, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:01:46.681782 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.673237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51490 (local address 127.25.31.190:42995)
0806 16:01:46.673470 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:46.673475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:46.673624 (+ 149us) server_negotiation.cc:407] Connection header received
0806 16:01:46.673717 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:46.673720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:46.673773 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:46.673876 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:46.674871 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.675604 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.676574 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.676939 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.677042 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.678518 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:46.678522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:46.678524 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:46.678547 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:46.680117 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:46.680528 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:46.680533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:46.680535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:46.680636 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:46.680912 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:46.680916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:46.680918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:46.681095 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:46.681308 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:46.681469 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 16:01:46.681599 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":76,"threads_started":1}
I20260806 16:01:46.684253 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.671150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:37737 (local address 127.25.31.190:42995)
0806 16:01:46.671283 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:46.671287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:46.672074 (+ 787us) server_negotiation.cc:407] Connection header received
0806 16:01:46.673947 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:46.673951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:46.674002 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:46.674093 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:46.675380 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.676025 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.677443 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.677805 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.677924 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.680480 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:46.680484 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:46.680487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:46.680509 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:46.682117 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:46.682750 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:46.682753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:46.682755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:46.682834 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:46.683194 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:46.683198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:46.683199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:46.683365 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:46.683547 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:46.683983 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 16:01:46.684093 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 16:01:46.684978 8624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.671520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:37737)
0806 16:01:46.671935 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:46.671976 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:46.673731 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:46.674242 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:46.674248 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:46.674662 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:46.675239 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.675257 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.676150 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.676153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.677322 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.677335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.677954 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.677956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.678092 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.678625 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:46.678640 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:46.680317 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:46.682237 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:46.682243 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:46.682255 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:46.682642 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:46.682932 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:46.682937 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:46.682947 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:46.683093 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:46.683485 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:46.683491 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:46.683742 (+ 251us) client_negotiation.cc:769] Sending connection context
0806 16:01:46.683918 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 16:01:46.684219 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":110,"threads_started":1}
I20260806 16:01:46.686039 8622 heartbeater.cc:347] Connected to a master server at 127.25.31.190:42995
I20260806 16:01:46.686233 8622 heartbeater.cc:464] Registering TS with master...
I20260806 16:01:46.686705 8622 heartbeater.cc:511] Master 127.25.31.190:42995 requested a full tablet report, sending...
I20260806 16:01:46.687610 8145 ts_manager.cc:194] Registered new tserver with Master: 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715)
I20260806 16:01:46.688144 8145 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:37737
I20260806 16:01:46.695242 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:01:46.699662 8341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.695794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51500 (local address 127.25.31.190:42995)
0806 16:01:46.695991 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:46.695996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:46.696192 (+ 196us) server_negotiation.cc:407] Connection header received
0806 16:01:46.696386 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:46.696389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:46.696482 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:46.696592 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:01:46.696839 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.697341 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.698215 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.698495 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.698604 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.698928 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:01:46.699278 (+ 350us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:01:46.699408 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:46.699474 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 16:01:46.699524 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 16:01:46.700245 8145 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:51500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:01:46.703426 8145 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:01:46.710829 8269 tablet_service.cc:1467] Processing CreateTablet for tablet 2e63cba14ea94beab89d508ef2c15e42 (DEFAULT_TABLE table=test-table [id=35d2bbad49044ce08ec7c79023c2c5b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:01:46.710851 8414 tablet_service.cc:1467] Processing CreateTablet for tablet 2e63cba14ea94beab89d508ef2c15e42 (DEFAULT_TABLE table=test-table [id=35d2bbad49044ce08ec7c79023c2c5b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:01:46.711411 8269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e63cba14ea94beab89d508ef2c15e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:01:46.711444 8414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e63cba14ea94beab89d508ef2c15e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:01:46.714092 8370 tablet_bootstrap.cc:492] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea: Bootstrap starting.
I20260806 16:01:46.714227 8226 tablet_bootstrap.cc:492] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4: Bootstrap starting.
I20260806 16:01:46.715117 8226 tablet_bootstrap.cc:654] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4: Neither blocks nor log segments found. Creating new log.
I20260806 16:01:46.715179 8370 tablet_bootstrap.cc:654] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea: Neither blocks nor log segments found. Creating new log.
I20260806 16:01:46.716359 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.709949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:36715 (local address 127.0.0.1:37842)
0806 16:01:46.710053 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:46.710069 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:46.710147 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:46.711088 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:46.711091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:46.711107 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:01:46.711216 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.711221 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.713031 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.713034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.714235 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.714249 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.715868 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.715874 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.715975 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.716030 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 16:01:46.716122 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 16:01:46.716220 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 16:01:46.716452 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.710192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37842 (local address 127.25.31.131:36715)
0806 16:01:46.710571 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:46.710583 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:46.710608 (+ 25us) server_negotiation.cc:407] Connection header received
0806 16:01:46.710682 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:46.710687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:46.710893 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:46.711366 (+ 473us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:01:46.711516 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.712874 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.714432 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.715725 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.715848 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.716007 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:46.716169 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 16:01:46.716282 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":162,"threads_started":1}
I20260806 16:01:46.716619 8370 tablet_bootstrap.cc:492] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea: No bootstrap required, opened a new log
I20260806 16:01:46.716715 8370 ts_tablet_manager.cc:1397] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:01:46.717127 8370 raft_consensus.cc:348] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:46.717229 8370 raft_consensus.cc:374] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:01:46.717257 8370 raft_consensus.cc:729] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0ce19b733cb444fadded86d8096d6ea, State: Initialized, Role: FOLLOWER
I20260806 16:01:46.717409 8370 consensus_queue.cc:260] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:46.717651 8370 ts_tablet_manager.cc:1428] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 16:01:46.718484 8226 tablet_bootstrap.cc:492] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4: No bootstrap required, opened a new log
I20260806 16:01:46.718565 8226 ts_tablet_manager.cc:1397] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 16:01:46.718976 8226 raft_consensus.cc:348] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:46.719082 8226 raft_consensus.cc:374] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:01:46.719126 8226 raft_consensus.cc:729] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2d44e8a19940b3a78c44e026d3f0e4, State: Initialized, Role: FOLLOWER
I20260806 16:01:46.719259 8226 consensus_queue.cc:260] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:46.719551 8226 ts_tablet_manager.cc:1428] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 16:01:46.719731 8553 tablet_service.cc:1467] Processing CreateTablet for tablet 2e63cba14ea94beab89d508ef2c15e42 (DEFAULT_TABLE table=test-table [id=35d2bbad49044ce08ec7c79023c2c5b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:01:46.720866 8553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e63cba14ea94beab89d508ef2c15e42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:01:46.727542 8634 tablet_bootstrap.cc:492] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1: Bootstrap starting.
I20260806 16:01:46.729775 8345 raft_consensus.cc:482] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:01:46.729856 8634 tablet_bootstrap.cc:654] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1: Neither blocks nor log segments found. Creating new log.
I20260806 16:01:46.729979 8345 raft_consensus.cc:504] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:46.730755 8634 log.cc:824] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:46.731230 8345 leader_election.cc:288] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715), a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557)
I20260806 16:01:46.733140 8634 tablet_bootstrap.cc:492] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1: No bootstrap required, opened a new log
I20260806 16:01:46.733343 8634 ts_tablet_manager.cc:1397] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 16:01:46.736274 8636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.731834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:34557 (local address 127.25.31.129:47719)
0806 16:01:46.732464 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:46.732478 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:46.732616 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:46.733024 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:46.733028 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:46.733047 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:01:46.733162 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.733168 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.734205 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.734208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.735092 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.735101 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.735794 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.735798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.735904 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.735950 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 16:01:46.736034 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 16:01:46.736103 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":1,"thread_start_us":104,"threads_started":1}
I20260806 16:01:46.736434 8342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.731542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:36715 (local address 127.25.31.129:40421)
0806 16:01:46.732107 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:46.732122 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:46.732220 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:46.732450 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:46.732453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:46.732471 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:01:46.732592 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.732598 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.733853 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.733856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.735152 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:46.735167 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.736045 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.736049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:46.736145 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.736165 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 16:01:46.736235 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 16:01:46.736307 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":25}
I20260806 16:01:46.736624 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.731659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:40421 (local address 127.25.31.131:36715)
0806 16:01:46.731801 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:46.731806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:46.732191 (+ 385us) server_negotiation.cc:407] Connection header received
0806 16:01:46.732312 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:46.732314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:46.732348 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:46.732437 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:01:46.732833 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.733732 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.735305 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.735884 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.736057 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.736107 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:46.736262 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 16:01:46.736332 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 16:01:46.736930 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.731911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:47719 (local address 127.25.31.130:34557)
0806 16:01:46.732033 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:46.732037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:46.732548 (+ 511us) server_negotiation.cc:407] Connection header received
0806 16:01:46.732748 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:46.732751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:46.732819 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:46.732904 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:01:46.733352 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.734033 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.735211 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.735660 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.735796 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.735933 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:46.736697 (+ 764us) server_negotiation.cc:299] Negotiation successful
0806 16:01:46.736794 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:01:46.737128 8573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e63cba14ea94beab89d508ef2c15e42" candidate_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" is_pre_election: true
I20260806 16:01:46.737290 8634 raft_consensus.cc:348] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:46.737448 8435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e63cba14ea94beab89d508ef2c15e42" candidate_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ce19b733cb444fadded86d8096d6ea" is_pre_election: true
I20260806 16:01:46.737548 8634 raft_consensus.cc:374] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:01:46.737598 8634 raft_consensus.cc:729] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ca9009352e64d1ab4a671e0c6cef7f1, State: Initialized, Role: FOLLOWER
I20260806 16:01:46.737725 8435 raft_consensus.cc:2454] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de2d44e8a19940b3a78c44e026d3f0e4 in term 0.
I20260806 16:01:46.738041 8634 consensus_queue.cc:260] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:46.738160 8222 leader_election.cc:302] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0ce19b733cb444fadded86d8096d6ea, de2d44e8a19940b3a78c44e026d3f0e4; no voters:
I20260806 16:01:46.738323 8573 raft_consensus.cc:2454] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de2d44e8a19940b3a78c44e026d3f0e4 in term 0.
I20260806 16:01:46.738492 8345 raft_consensus.cc:2781] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:01:46.738595 8345 raft_consensus.cc:482] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:01:46.738654 8345 raft_consensus.cc:3037] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:01:46.739323 8634 ts_tablet_manager.cc:1428] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 16:01:46.739791 8622 heartbeater.cc:503] Master 127.25.31.190:42995 was elected leader, sending a full tablet report...
I20260806 16:01:46.739794 8345 raft_consensus.cc:504] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
W20260806 16:01:46.740008 8484 tablet.cc:2369] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:01:46.740104 8345 leader_election.cc:288] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 1 election: Requested vote from peers 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715), a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557)
I20260806 16:01:46.740429 8573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e63cba14ea94beab89d508ef2c15e42" candidate_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1"
I20260806 16:01:46.740546 8573 raft_consensus.cc:3037] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:01:46.740772 8435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e63cba14ea94beab89d508ef2c15e42" candidate_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0ce19b733cb444fadded86d8096d6ea"
I20260806 16:01:46.740903 8435 raft_consensus.cc:3037] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:01:46.741762 8573 raft_consensus.cc:2454] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de2d44e8a19940b3a78c44e026d3f0e4 in term 1.
I20260806 16:01:46.742034 8222 leader_election.cc:302] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ca9009352e64d1ab4a671e0c6cef7f1, de2d44e8a19940b3a78c44e026d3f0e4; no voters:
I20260806 16:01:46.742043 8435 raft_consensus.cc:2454] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de2d44e8a19940b3a78c44e026d3f0e4 in term 1.
I20260806 16:01:46.742201 8345 raft_consensus.cc:2781] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:01:46.742425 8345 raft_consensus.cc:686] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 LEADER]: Becoming Leader. State: Replica: de2d44e8a19940b3a78c44e026d3f0e4, State: Running, Role: LEADER
I20260806 16:01:46.742674 8345 consensus_queue.cc:237] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:46.745254 8144 catalog_manager.cc:5495] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to de2d44e8a19940b3a78c44e026d3f0e4 (127.25.31.129). New cstate: current_term: 1 leader_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } health_report { overall_health: HEALTHY } } }
I20260806 16:01:46.756042 8347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.752814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56758 (local address 127.25.31.129:43861)
0806 16:01:46.752926 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:46.752929 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:46.752941 (+ 12us) server_negotiation.cc:407] Connection header received
0806 16:01:46.753061 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:46.753063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:46.753108 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:46.753192 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:01:46.753383 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.753872 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.754730 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:46.754960 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:46.755036 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:46.755256 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:01:46.755657 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:01:46.755777 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:46.755856 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 16:01:46.755902 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 16:01:46.759624 8435 raft_consensus.cc:1261] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea [term 1 FOLLOWER]: Refusing update from remote peer de2d44e8a19940b3a78c44e026d3f0e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:01:46.759649 8573 raft_consensus.cc:1261] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 1 FOLLOWER]: Refusing update from remote peer de2d44e8a19940b3a78c44e026d3f0e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:01:46.760257 8345 consensus_queue.cc:1035] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:01:46.760458 8638 consensus_queue.cc:1035] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:01:46.766345 8640 mvcc.cc:205] Tried to move back new op lower bound from 7315587509281722368 to 7315587509220659200. Current Snapshot: MvccSnapshot[applied={T|T < 7315587509281722368}]
I20260806 16:01:46.768304 8641 mvcc.cc:205] Tried to move back new op lower bound from 7315587509281722368 to 7315587509220659200. Current Snapshot: MvccSnapshot[applied={T|T < 7315587509281722368}]
I20260806 16:01:46.774883 8144 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:51500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:01:46.775053 8144 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:51500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:01:46.777354 8144 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: Sending DeleteTablet for 3 replicas of tablet 2e63cba14ea94beab89d508ef2c15e42
I20260806 16:01:46.778062 8269 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e63cba14ea94beab89d508ef2c15e42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:01:46 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:56754
I20260806 16:01:46.778062 8414 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e63cba14ea94beab89d508ef2c15e42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:01:46 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:38886
I20260806 16:01:46.778229 8489 tablet_replica.cc:330] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea: stopping tablet replica
I20260806 16:01:46.778229 8348 tablet_replica.cc:330] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4: stopping tablet replica
I20260806 16:01:46.778198 8553 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e63cba14ea94beab89d508ef2c15e42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:01:46 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:37842
I20260806 16:01:46.778357 8348 raft_consensus.cc:2229] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:01:46.778431 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 8183
I20260806 16:01:46.778460 8489 raft_consensus.cc:2229] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:01:46.778580 8489 raft_consensus.cc:2258] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:01:46.778618 8348 raft_consensus.cc:2258] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:01:46.781600 8649 tablet_replica.cc:330] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1: stopping tablet replica
I20260806 16:01:46.781883 8489 ts_tablet_manager.cc:1905] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:01:46.781934 8649 raft_consensus.cc:2229] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:01:46.782229 8649 raft_consensus.cc:2258] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:01:46.783772 8649 ts_tablet_manager.cc:1905] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:01:46.787201 8649 ts_tablet_manager.cc:1918] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:01:46.787233 8489 ts_tablet_manager.cc:1918] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:01:46.787312 8489 log.cc:1196] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/2e63cba14ea94beab89d508ef2c15e42
I20260806 16:01:46.787320 8649 log.cc:1196] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/2e63cba14ea94beab89d508ef2c15e42
I20260806 16:01:46.787608 8489 ts_tablet_manager.cc:1939] T 2e63cba14ea94beab89d508ef2c15e42 P a0ce19b733cb444fadded86d8096d6ea: Deleting consensus metadata
I20260806 16:01:46.787684 8649 ts_tablet_manager.cc:1939] T 2e63cba14ea94beab89d508ef2c15e42 P 3ca9009352e64d1ab4a671e0c6cef7f1: Deleting consensus metadata
W20260806 16:01:46.787833 8132 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:43861 (error 108)
I20260806 16:01:46.788105 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 8343
I20260806 16:01:46.791076 8131 catalog_manager.cc:4841] TS 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715): tablet 2e63cba14ea94beab89d508ef2c15e42 (table test-table [id=35d2bbad49044ce08ec7c79023c2c5b8]) successfully deleted
I20260806 16:01:46.791428 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.791078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:43861 (local address 127.0.0.1:56770)
0806 16:01:46.791261 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:46.791338 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:43861: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260806 16:01:46.791602 8132 catalog_manager.cc:4584] TS de2d44e8a19940b3a78c44e026d3f0e4 (127.25.31.129:43861): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e63cba14ea94beab89d508ef2c15e42: Network error: Client connection negotiation failed: client connection to 127.25.31.129:43861: connect: Connection refused (error 111)
I20260806 16:01:46.796557 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 8486
I20260806 16:01:46.798091 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:46.796633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:34557 (local address 127.0.0.1:38894)
0806 16:01:46.797927 (+ 1294us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:46.797994 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.130:34557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1187}
I20260806 16:01:46.803138 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 8112
I20260806 16:01:46.809849 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42995
--webserver_interface=127.25.31.190
--webserver_port=42093
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:42995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:01:46.920030 8651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:46.920287 8651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:46.920327 8651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:46.924386 8651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:01:46.924443 8651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:46.924463 8651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:46.924479 8651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:01:46.924522 8651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:01:46.930040 8651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:42995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=42093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:46.931105 8651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:46.932103 8651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:46.940625 8659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:46.940687 8651 server_base.cc:1034] running on GCE node
W20260806 16:01:46.940593 8657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:46.940606 8656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:46.941475 8651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:46.942559 8651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:46.943717 8651 hybrid_clock.cc:648] HybridClock initialized: now 1786032106943705 us; error 25 us; skew 500 ppm
Aug 06 16:01:46 dist-test-slave-tm1g krb5kdc[7553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032106, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:46.946494 8651 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:01:46.947266 8651 webserver.cc:460] Webserver started at http://127.25.31.190:42093/ using document root <none> and password file <none>
I20260806 16:01:46.948012 8651 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:46.948084 8651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:46.951777 8651 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 16:01:46.953758 8666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:46.955019 8651 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:01:46.955152 8651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "34726477f969480d9cdbd4d4d130722d"
format_stamp: "Formatted at 2026-08-06 16:01:44 on dist-test-slave-tm1g"
server_key: "4f97f9b394d7ff6e02324198e09ca6ac"
server_key_iv: "3e94a155dd0c35ce33b3097ae6b9ffe6"
server_key_version: "encryptionkey@0"
I20260806 16:01:46.955700 8651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:46.974182 8651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:46.976866 8651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:46.977061 8651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:46.984174 8651 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:42995
I20260806 16:01:46.984465 8719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:42995 every 8 connection(s)
I20260806 16:01:46.985239 8651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:01:46.986002 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 8651
I20260806 16:01:46.986478 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:43861
--local_ip_for_outbound_sockets=127.25.31.129
--tserver_master_addrs=127.25.31.190:42995
--webserver_port=39275
--webserver_interface=127.25.31.129
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:01:46.989974 8720 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:01:46.991587 8720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: Bootstrap starting.
I20260806 16:01:47.009152 8720 log.cc:824] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:47.020264 8720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:01:47.020666 8720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d: Bootstrap complete.
I20260806 16:01:47.024693 8720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } }
I20260806 16:01:47.025068 8720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34726477f969480d9cdbd4d4d130722d, State: Initialized, Role: FOLLOWER
I20260806 16:01:47.025563 8720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } }
I20260806 16:01:47.025673 8720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:01:47.025756 8720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:01:47.025873 8720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 2 FOLLOWER]: Advancing to term 3
I20260806 16:01:47.027745 8720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } }
I20260806 16:01:47.028064 8720 leader_election.cc:302] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34726477f969480d9cdbd4d4d130722d; no voters:
I20260806 16:01:47.028389 8720 leader_election.cc:288] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 16:01:47.028471 8724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 3 FOLLOWER]: Leader election won for term 3
I20260806 16:01:47.028678 8724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [term 3 LEADER]: Becoming Leader. State: Replica: 34726477f969480d9cdbd4d4d130722d, State: Running, Role: LEADER
I20260806 16:01:47.028978 8724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } }
I20260806 16:01:47.029621 8720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:01:47.030452 8726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "34726477f969480d9cdbd4d4d130722d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } } }
I20260806 16:01:47.030583 8726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [sys.catalog]: This master's current role is: LEADER
I20260806 16:01:47.030256 8725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34726477f969480d9cdbd4d4d130722d. Latest consensus state: current_term: 3 leader_uuid: "34726477f969480d9cdbd4d4d130722d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34726477f969480d9cdbd4d4d130722d" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42995 } } }
I20260806 16:01:47.031239 8725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34726477f969480d9cdbd4d4d130722d [sys.catalog]: This master's current role is: LEADER
I20260806 16:01:47.031507 8733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:01:47.036435 8733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:01:47.037288 8733 catalog_manager.cc:1259] Loaded cluster ID: e00d6c0465a0420f91e5fcf1f67e9005
I20260806 16:01:47.037351 8733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 16:01:47.038321 8733 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd59c4849ad google::LogMessage::Fail() at ??:0
@ 0x7fd59c48788a google::LogMessage::SendToLog() at ??:0
@ 0x7fd59c4844de google::LogMessage::Flush() at ??:0
@ 0x7fd59c485a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd5a6dc7fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fd5a6dc8d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd5a6dc5579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fd5a6e09640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd59e6d63d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd59e6d6c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fd59e6d83f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd59e6c7512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd59fd0b6db start_thread at ??:0
@ 0x7fd59af8261f clone at ??:0
W20260806 16:01:47.117121 8722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:47.117384 8722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:47.117502 8722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:47.121564 8722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:47.121646 8722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:47.121754 8722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:01:47.127135 8722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:43861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=39275
--tserver_master_addrs=127.25.31.190:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:47.128243 8722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:47.129278 8722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:47.138861 8765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:47.138892 8764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:47.138914 8767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:47.139243 8722 server_base.cc:1034] running on GCE node
I20260806 16:01:47.139680 8722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:47.140419 8722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:47.141623 8722 hybrid_clock.cc:648] HybridClock initialized: now 1786032107141600 us; error 32 us; skew 500 ppm
Aug 06 16:01:47 dist-test-slave-tm1g krb5kdc[7553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032107, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:47.145099 8722 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 16:01:47.146178 8722 webserver.cc:460] Webserver started at http://127.25.31.129:39275/ using document root <none> and password file <none>
I20260806 16:01:47.147023 8722 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:47.147133 8722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:47.152190 8722 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 16:01:47.154850 8774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:47.156142 8722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:01:47.156277 8722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "de2d44e8a19940b3a78c44e026d3f0e4"
format_stamp: "Formatted at 2026-08-06 16:01:45 on dist-test-slave-tm1g"
server_key: "116a2e3259c5d17a5a040f54e369939c"
server_key_iv: "23ee6936d4cd0ed0afd98950ab35f3a8"
server_key_version: "encryptionkey@0"
I20260806 16:01:47.156810 8722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:47.169919 8722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:47.173286 8722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:47.173501 8722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:47.174161 8722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:01:47.175325 8781 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 16:01:47.180940 8722 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 16:01:47.181033 8722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260806 16:01:47.181120 8722 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 16:01:47.182791 8722 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 16:01:47.182862 8722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 16:01:47.183208 8781 tablet_bootstrap.cc:492] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4: Bootstrap starting.
I20260806 16:01:47.193727 8722 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:43861
I20260806 16:01:47.195264 8722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 16:01:47.196172 8892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:43861 every 8 connection(s)
I20260806 16:01:47.205355 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 8722
I20260806 16:01:47.205775 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:34557
--local_ip_for_outbound_sockets=127.25.31.130
--tserver_master_addrs=127.25.31.190:42995
--webserver_port=33749
--webserver_interface=127.25.31.130
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:01:47.205906 8781 log.cc:824] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4: Log is configured to *not* fsync() on all Append() calls
I20260806 16:01:47.215170 8781 tablet_bootstrap.cc:492] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:01:47.215611 8781 tablet_bootstrap.cc:492] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4: Bootstrap complete.
I20260806 16:01:47.217150 8781 ts_tablet_manager.cc:1397] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4: Time spent bootstrapping tablet: real 0.034s user 0.026s sys 0.005s
I20260806 16:01:47.220606 8781 raft_consensus.cc:348] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:47.221158 8781 raft_consensus.cc:729] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2d44e8a19940b3a78c44e026d3f0e4, State: Initialized, Role: FOLLOWER
I20260806 16:01:47.221693 8781 consensus_queue.cc:260] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:47.222746 8781 ts_tablet_manager.cc:1428] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260806 16:01:47.274427 8895 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:01:47.197957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:59853)
0806 16:01:47.198472 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:47.198525 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:47.199323 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:47.273317 (+ 73994us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":148,"threads_started":1}
W20260806 16:01:47.274956 8893 heartbeater.cc:644] Failed to heartbeat to 127.25.31.190:42995 (0 consecutive failures): Network error: Failed to ping master at 127.25.31.190:42995: Client connection negotiation failed: client connection to 127.25.31.190:42995: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 16:01:47.275806 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:47.275488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:43029)
0806 16:01:47.275656 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:47.275735 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 16:01:47.276511 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:47.276272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:58331)
0806 16:01:47.276394 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:47.276453 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 16:01:47.276713 8893 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:01:47.318832 8898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:47.319072 8898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:47.319113 8898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:47.322919 8898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:47.322962 8898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:47.323022 8898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:01:47.328020 8898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:34557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=33749
--tserver_master_addrs=127.25.31.190:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:47.329032 8898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:47.329943 8898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:47.339178 8904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:47.339190 8907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:47.339195 8905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:47.339588 8898 server_base.cc:1034] running on GCE node
I20260806 16:01:47.339973 8898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:47.340579 8898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:47.341740 8898 hybrid_clock.cc:648] HybridClock initialized: now 1786032107341727 us; error 25 us; skew 500 ppm
Aug 06 16:01:47 dist-test-slave-tm1g krb5kdc[7553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032107, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:47.344353 8898 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 16:01:47.345206 8898 webserver.cc:460] Webserver started at http://127.25.31.130:33749/ using document root <none> and password file <none>
I20260806 16:01:47.345767 8898 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:47.345860 8898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:47.349632 8898 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:01:47.351678 8914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:47.352849 8898 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:01:47.352978 8898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "a0ce19b733cb444fadded86d8096d6ea"
format_stamp: "Formatted at 2026-08-06 16:01:45 on dist-test-slave-tm1g"
server_key: "4ab8b7015fd2738a4fa954bf8838ecfe"
server_key_iv: "88fcfac14de65bc17745e8b0b615cc17"
server_key_version: "encryptionkey@0"
I20260806 16:01:47.353487 8898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:47.367882 8898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:47.370553 8898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:47.370743 8898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:47.371299 8898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:01:47.372148 8898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:01:47.372195 8898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:47.372262 8898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:01:47.372303 8898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:47.381076 8898 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:34557
I20260806 16:01:47.381114 9033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:34557 every 8 connection(s)
I20260806 16:01:47.382017 8898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:01:47.382324 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 8898
I20260806 16:01:47.382819 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:36715
--local_ip_for_outbound_sockets=127.25.31.131
--tserver_master_addrs=127.25.31.190:42995
--webserver_port=35433
--webserver_interface=127.25.31.131
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:01:47.386031 9037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:47.384212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:44787)
0806 16:01:47.384729 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:47.384832 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":163,"threads_started":1}
W20260806 16:01:47.386504 9034 heartbeater.cc:644] Failed to heartbeat to 127.25.31.190:42995 (0 consecutive failures): Network error: Failed to ping master at 127.25.31.190:42995: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
I20260806 16:01:47.387099 9037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:47.386867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:43075)
0806 16:01:47.386987 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:47.387038 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 16:01:47.387799 9037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:47.387614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:35493)
0806 16:01:47.387713 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:47.387753 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 16:01:47.388038 9034 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:01:47.493350 9036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:01:47.493592 9036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:01:47.493638 9036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:01:47.497563 9036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:01:47.497614 9036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:01:47.497710 9036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:01:47.502833 9036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:36715
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=35433
--tserver_master_addrs=127.25.31.190:42995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:01:47.503820 9036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:01:47.504796 9036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:01:47.514215 9043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:47.514211 9042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:01:47.514257 9045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:01:47.514509 9036 server_base.cc:1034] running on GCE node
I20260806 16:01:47.514974 9036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:01:47.515605 9036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:01:47.516783 9036 hybrid_clock.cc:648] HybridClock initialized: now 1786032107516759 us; error 46 us; skew 500 ppm
Aug 06 16:01:47 dist-test-slave-tm1g krb5kdc[7553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032107, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:01:47.519457 9036 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 16:01:47.520311 9036 webserver.cc:460] Webserver started at http://127.25.31.131:35433/ using document root <none> and password file <none>
I20260806 16:01:47.520849 9036 fs_manager.cc:359] Metadata directory not provided
I20260806 16:01:47.520951 9036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:01:47.524648 9036 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 16:01:47.526621 9052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:47.527755 9036 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 16:01:47.527890 9036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "3ca9009352e64d1ab4a671e0c6cef7f1"
format_stamp: "Formatted at 2026-08-06 16:01:45 on dist-test-slave-tm1g"
server_key: "223c7f5951332516670398b34caf134c"
server_key_iv: "4765abb0c3a2c73a4de9074576071b3b"
server_key_version: "encryptionkey@0"
I20260806 16:01:47.528411 9036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:01:47.557950 9036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:01:47.560864 9036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:01:47.561077 9036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:01:47.561722 9036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:01:47.562605 9036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:01:47.562678 9036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:47.562763 9036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:01:47.562808 9036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:01:47.571810 9036 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:36715
I20260806 16:01:47.571834 9172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:36715 every 8 connection(s)
I20260806 16:01:47.572755 9036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:01:47.576630 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:47.574919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:55869)
0806 16:01:47.575392 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:47.575505 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":119,"threads_started":1}
W20260806 16:01:47.577055 9173 heartbeater.cc:644] Failed to heartbeat to 127.25.31.190:42995 (0 consecutive failures): Network error: Failed to ping master at 127.25.31.190:42995: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
I20260806 16:01:47.577661 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:47.577437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:47415)
0806 16:01:47.577549 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:47.577599 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 16:01:47.578218 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:47.578017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:42119)
0806 16:01:47.578121 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:47.578164 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 16:01:47.578423 9173 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 16:01:47.579989 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 9036
I20260806 16:01:48.277699 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:48.277273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:36973)
0806 16:01:48.277542 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:48.277594 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":78,"threads_started":1}
I20260806 16:01:48.388962 9178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:48.388569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:40341)
0806 16:01:48.388807 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:48.388864 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":91,"threads_started":1}
I20260806 16:01:48.579429 9179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:48.579027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:35517)
0806 16:01:48.579266 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:48.579325 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260806 16:01:48.897673 9180 raft_consensus.cc:482] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:01:48.897894 9180 raft_consensus.cc:504] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:48.899097 9180 leader_election.cc:288] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715), a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557)
Aug 06 16:01:48 dist-test-slave-tm1g krb5kdc[7553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032107, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.130@KRBTEST.COM
Aug 06 16:01:48 dist-test-slave-tm1g krb5kdc[7553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032107, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.131@KRBTEST.COM
I20260806 16:01:48.913435 9181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:48.899302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:36715 (local address 127.25.31.129:50573)
0806 16:01:48.899585 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:48.899632 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:48.899750 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:48.901375 (+ 1625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:48.901380 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:48.901965 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:48.902484 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:48.902500 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:48.903788 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:48.903790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:48.904683 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:48.904692 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:48.905385 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:48.905388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:48.905614 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:48.906280 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:48.906296 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:48.908467 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:48.910716 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:48.910720 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:48.910723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:48.911023 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:48.911363 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:48.911365 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:48.911367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:48.911522 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:48.912274 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:48.912281 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:48.912657 (+ 376us) client_negotiation.cc:769] Sending connection context
0806 16:01:48.912884 (+ 227us) client_negotiation.cc:240] Negotiation successful
0806 16:01:48.913265 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":334,"spinlock_wait_cycles":16000,"thread_start_us":91,"threads_started":1}
I20260806 16:01:48.913435 9184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:48.899664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:34557 (local address 127.25.31.129:54219)
0806 16:01:48.899994 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:48.900007 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:01:48.900088 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:01:48.901474 (+ 1386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:01:48.901477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:01:48.901977 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:01:48.902485 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:48.902502 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:48.903786 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:48.903789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:48.904685 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:01:48.904694 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:01:48.905405 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:01:48.905409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:01:48.905613 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:48.906383 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:01:48.906400 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:01:48.908100 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:01:48.910511 (+ 2411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:48.910515 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:48.910526 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:48.910906 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:48.911293 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:01:48.911295 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:01:48.911296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:01:48.911522 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:01:48.912274 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:01:48.912281 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:01:48.912657 (+ 376us) client_negotiation.cc:769] Sending connection context
0806 16:01:48.912884 (+ 227us) client_negotiation.cc:240] Negotiation successful
0806 16:01:48.913211 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":190,"spinlock_wait_cycles":32384,"thread_start_us":70,"threads_started":1}
I20260806 16:01:48.913499 9185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:48.900002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:54219 (local address 127.25.31.130:34557)
0806 16:01:48.900323 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:48.900333 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:48.900364 (+ 31us) server_negotiation.cc:407] Connection header received
0806 16:01:48.900977 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:48.900982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:48.901253 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:48.901535 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:48.902669 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:48.903627 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:48.904811 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:48.905263 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:48.905448 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:48.908329 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:48.908334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:48.908346 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:48.908372 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:48.910400 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:48.911023 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:48.911027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:48.911031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:48.911197 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:48.911636 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:48.911640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:48.911642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:48.912153 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:48.912289 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:48.913033 (+ 744us) server_negotiation.cc:299] Negotiation successful
0806 16:01:48.913311 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":100,"threads_started":1}
I20260806 16:01:48.914065 9182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:48.899502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:50573 (local address 127.25.31.131:36715)
0806 16:01:48.899839 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0806 16:01:48.899874 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0806 16:01:48.899922 (+ 48us) server_negotiation.cc:407] Connection header received
0806 16:01:48.900676 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:01:48.900682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:01:48.901016 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:01:48.901324 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:01:48.902662 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:48.903626 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:48.904813 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:01:48.905259 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:01:48.905437 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:01:48.908655 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:01:48.908664 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:01:48.908681 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:01:48.908719 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:01:48.910615 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:48.911125 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:48.911130 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:48.911134 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:48.911269 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:01:48.911636 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:01:48.911640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:01:48.911642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:01:48.912153 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:01:48.912289 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 16:01:48.913033 (+ 744us) server_negotiation.cc:299] Negotiation successful
0806 16:01:48.913311 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":74,"threads_started":1}
W20260806 16:01:48.915848 8777 leader_election.cc:341] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557): Not found: Tablet not found: 2e63cba14ea94beab89d508ef2c15e42
W20260806 16:01:48.916038 8777 leader_election.cc:341] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715): Not found: Tablet not found: 2e63cba14ea94beab89d508ef2c15e42
I20260806 16:01:48.916082 8777 leader_election.cc:302] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de2d44e8a19940b3a78c44e026d3f0e4; no voters: 3ca9009352e64d1ab4a671e0c6cef7f1, a0ce19b733cb444fadded86d8096d6ea
I20260806 16:01:48.916292 9180 raft_consensus.cc:2726] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:01:49.278927 9181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:49.278587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:59649)
0806 16:01:49.278747 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:49.278829 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 16:01:49.390028 9186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:49.389595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:53895)
0806 16:01:49.389868 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:49.389925 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":88,"threads_started":1}
I20260806 16:01:49.580583 9187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:49.580229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:58163)
0806 16:01:49.580450 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:49.580498 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":72,"threads_started":1}
I20260806 16:01:50.280190 9188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:50.279739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:35449)
0806 16:01:50.280028 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:50.280089 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":84,"threads_started":1}
I20260806 16:01:50.391201 9189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:50.390788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:43085)
0806 16:01:50.391049 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:50.391108 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":84,"threads_started":1}
I20260806 16:01:50.430130 9190 raft_consensus.cc:482] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:01:50.430349 9190 raft_consensus.cc:504] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:50.430807 9190 leader_election.cc:288] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715), a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557)
W20260806 16:01:50.431705 8777 leader_election.cc:341] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715): Not found: Tablet not found: 2e63cba14ea94beab89d508ef2c15e42
W20260806 16:01:50.431866 8777 leader_election.cc:341] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557): Not found: Tablet not found: 2e63cba14ea94beab89d508ef2c15e42
I20260806 16:01:50.431939 8777 leader_election.cc:302] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de2d44e8a19940b3a78c44e026d3f0e4; no voters: 3ca9009352e64d1ab4a671e0c6cef7f1, a0ce19b733cb444fadded86d8096d6ea
I20260806 16:01:50.432106 9190 raft_consensus.cc:2726] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:01:50.581828 9191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:50.581354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:53327)
0806 16:01:50.581648 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:50.581728 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":129,"threads_started":1}
I20260806 16:01:51.281396 9192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:51.280973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:50893)
0806 16:01:51.281242 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:51.281297 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":78,"threads_started":1}
I20260806 16:01:51.392498 9194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:51.391945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:45293)
0806 16:01:51.392275 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:51.392350 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":86,"threads_started":1}
I20260806 16:01:51.583027 9195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:51.582614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:40743)
0806 16:01:51.582886 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:51.582941 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":73,"threads_started":1}
I20260806 16:01:52.282912 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:52.282381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:36503)
0806 16:01:52.282739 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:52.282801 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":94,"threads_started":1}
I20260806 16:01:52.393662 9197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:52.393266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:45715)
0806 16:01:52.393512 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:52.393570 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
I20260806 16:01:52.584264 9198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:52.583819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:44293)
0806 16:01:52.584104 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:52.584161 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":93,"threads_started":1}
I20260806 16:01:52.978847 9199 raft_consensus.cc:482] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:01:52.979029 9199 raft_consensus.cc:504] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:52.979499 9199 leader_election.cc:288] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715), a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557)
W20260806 16:01:52.980370 8777 leader_election.cc:341] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715): Not found: Tablet not found: 2e63cba14ea94beab89d508ef2c15e42
W20260806 16:01:52.980533 8777 leader_election.cc:341] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557): Not found: Tablet not found: 2e63cba14ea94beab89d508ef2c15e42
I20260806 16:01:52.980598 8777 leader_election.cc:302] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de2d44e8a19940b3a78c44e026d3f0e4; no voters: 3ca9009352e64d1ab4a671e0c6cef7f1, a0ce19b733cb444fadded86d8096d6ea
I20260806 16:01:52.980747 9199 raft_consensus.cc:2726] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:01:53.284183 9200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:53.283727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:59167)
0806 16:01:53.284006 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:53.284066 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":80,"threads_started":1}
I20260806 16:01:53.394913 9201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:53.394486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:37717)
0806 16:01:53.394758 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:53.394819 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":74,"threads_started":1}
I20260806 16:01:53.585457 9202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:53.585020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:59643)
0806 16:01:53.585292 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:53.585363 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":91,"threads_started":1}
I20260806 16:01:54.285501 9203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:54.285032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:49967)
0806 16:01:54.285340 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:54.285400 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":105,"threads_started":1}
I20260806 16:01:54.396174 9204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:54.395655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:46695)
0806 16:01:54.396011 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:54.396079 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":75,"threads_started":1}
I20260806 16:01:54.586812 9205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:54.586220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:45703)
0806 16:01:54.586656 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:54.586716 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":193,"threads_started":1}
I20260806 16:01:55.286726 9206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:55.286231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:45325)
0806 16:01:55.286555 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:55.286611 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":129,"threads_started":1}
I20260806 16:01:55.396659 9207 raft_consensus.cc:482] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:01:55.396829 9207 raft_consensus.cc:504] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:55.397276 9207 leader_election.cc:288] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715), a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557)
I20260806 16:01:55.397432 9208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:55.396959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:57861)
0806 16:01:55.397218 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:55.397307 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":98,"threads_started":1}
W20260806 16:01:55.398102 8777 leader_election.cc:341] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557): Not found: Tablet not found: 2e63cba14ea94beab89d508ef2c15e42
W20260806 16:01:55.398211 8777 leader_election.cc:341] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715): Not found: Tablet not found: 2e63cba14ea94beab89d508ef2c15e42
I20260806 16:01:55.398248 8777 leader_election.cc:302] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de2d44e8a19940b3a78c44e026d3f0e4; no voters: 3ca9009352e64d1ab4a671e0c6cef7f1, a0ce19b733cb444fadded86d8096d6ea
I20260806 16:01:55.398432 9207 raft_consensus.cc:2726] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:01:55.587953 9209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:55.587541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:47775)
0806 16:01:55.587811 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:55.587865 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":113,"threads_started":1}
I20260806 16:01:56.287914 9210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:56.287491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:40909)
0806 16:01:56.287758 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:56.287814 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":96,"threads_started":1}
I20260806 16:01:56.398797 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:56.398398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:40855)
0806 16:01:56.398645 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:56.398701 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":78,"threads_started":1}
I20260806 16:01:56.589092 9212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:56.588666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:40505)
0806 16:01:56.588943 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:56.588998 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":92,"threads_started":1}
I20260806 16:01:57.289419 9213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:57.288944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:40397)
0806 16:01:57.289206 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:57.289260 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":98,"threads_started":1}
I20260806 16:01:57.399919 9214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:57.399526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:47681)
0806 16:01:57.399761 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:57.399813 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":75,"threads_started":1}
I20260806 16:01:57.590252 9215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:57.589860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:44779)
0806 16:01:57.590104 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:57.590159 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":72,"threads_started":1}
I20260806 16:01:58.290769 9216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:58.290259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:54231)
0806 16:01:58.290603 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:58.290663 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":155,"threads_started":1}
I20260806 16:01:58.401278 9217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:58.400743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:43595)
0806 16:01:58.401079 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:58.401149 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":79,"threads_started":1}
I20260806 16:01:58.591480 9218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:58.591062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:42443)
0806 16:01:58.591324 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:58.591381 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":73,"threads_started":1}
I20260806 16:01:59.292049 9219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:59.291551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:35007)
0806 16:01:59.291874 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:59.291940 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":85,"threads_started":1}
I20260806 16:01:59.402650 9220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:59.402102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:50043)
0806 16:01:59.402500 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:59.402556 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":82,"threads_started":1}
I20260806 16:01:59.592643 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:01:59.592200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:38741)
0806 16:01:59.592491 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 16:01:59.592547 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":102,"threads_started":1}
I20260806 16:01:59.688441 9222 raft_consensus.cc:482] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:01:59.688582 9222 raft_consensus.cc:504] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ca9009352e64d1ab4a671e0c6cef7f1" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36715 } } peers { permanent_uuid: "a0ce19b733cb444fadded86d8096d6ea" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34557 } } peers { permanent_uuid: "de2d44e8a19940b3a78c44e026d3f0e4" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43861 } }
I20260806 16:01:59.688997 9222 leader_election.cc:288] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715), a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557)
W20260806 16:01:59.689860 8777 leader_election.cc:341] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a0ce19b733cb444fadded86d8096d6ea (127.25.31.130:34557): Not found: Tablet not found: 2e63cba14ea94beab89d508ef2c15e42
W20260806 16:01:59.689965 8777 leader_election.cc:341] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ca9009352e64d1ab4a671e0c6cef7f1 (127.25.31.131:36715): Not found: Tablet not found: 2e63cba14ea94beab89d508ef2c15e42
I20260806 16:01:59.690034 8777 leader_election.cc:302] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de2d44e8a19940b3a78c44e026d3f0e4; no voters: 3ca9009352e64d1ab4a671e0c6cef7f1, a0ce19b733cb444fadded86d8096d6ea
I20260806 16:01:59.690181 9222 raft_consensus.cc:2726] T 2e63cba14ea94beab89d508ef2c15e42 P de2d44e8a19940b3a78c44e026d3f0e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:02:00.293896 9223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:00.293219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:35329)
0806 16:02:00.293657 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:00.293753 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":143,"threads_started":1}
I20260806 16:02:00.403820 9224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:00.403413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:43477)
0806 16:02:00.403667 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:00.403722 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":80,"threads_started":1}
I20260806 16:02:00.593962 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:00.593516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:34147)
0806 16:02:00.593789 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:00.593862 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":84,"threads_started":1}
I20260806 16:02:01.295296 9226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:01.294830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:36421)
0806 16:02:01.295130 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:01.295191 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":84,"threads_started":1}
I20260806 16:02:01.405040 9227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:01.404588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:50121)
0806 16:02:01.404881 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:01.404941 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":81,"threads_started":1}
I20260806 16:02:01.595207 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:01.594754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.131:57503)
0806 16:02:01.595058 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:01.595113 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":98,"threads_started":1}
I20260806 16:02:02.296696 9229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:02.296274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.129:54457)
0806 16:02:02.296542 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:02.296597 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":89,"threads_started":1}
I20260806 16:02:02.406515 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:02.405952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42995 (local address 127.25.31.130:49849)
0806 16:02:02.406191 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:02.406249 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":75,"threads_started":1}
I20260806 16:02:02.582008 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 8722
I20260806 16:02:02.589131 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 8898
I20260806 16:02:02.595029 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 9036
2026-08-06T16:02:02Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18968 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:02:02 dist-test-slave-tm1g krb5kdc[9233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:02:02 dist-test-slave-tm1g krb5kdc[9233](info): set up 2 sockets
Aug 06 16:02:02 dist-test-slave-tm1g krb5kdc[9233](info): commencing operation
krb5kdc: starting...
W20260806 16:02:03.655471 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:02:03 dist-test-slave-tm1g krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:02:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:02:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:02:03.782033 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:37419
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:37419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:02:03.890089 9247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:02:03.890398 9247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:02:03.890477 9247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:02:03.894429 9247 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 16:02:03.894513 9247 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 16:02:03.894541 9247 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 16:02:03.894572 9247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:02:03.894613 9247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:02:03.894656 9247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:02:03.894693 9247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:02:03.894726 9247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:02:03.899844 9247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:37419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:37419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:02:03.900862 9247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:02:03.901858 9247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:02:03.910704 9253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:03.910763 9247 server_base.cc:1034] running on GCE node
W20260806 16:02:03.910719 9252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:03.910717 9255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:03.911396 9247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:02:03.912500 9247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:02:03.913705 9247 hybrid_clock.cc:648] HybridClock initialized: now 1786032123913696 us; error 29 us; skew 500 ppm
Aug 06 16:02:03 dist-test-slave-tm1g krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032123, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:02:03.916623 9247 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:02:03.917555 9247 webserver.cc:460] Webserver started at http://127.25.31.190:35281/ using document root <none> and password file <none>
I20260806 16:02:03.918114 9247 fs_manager.cc:359] Metadata directory not provided
I20260806 16:02:03.918203 9247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:02:03.918488 9247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:02:03.920502 9247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "548f7867ad1e4b3fae33485cc0241dbc"
format_stamp: "Formatted at 2026-08-06 16:02:03 on dist-test-slave-tm1g"
server_key: "f46fee9c724fd4d131d4de27b0de32a7"
server_key_iv: "2cc28f646ba2d23b36d279a0e4876f11"
server_key_version: "encryptionkey@0"
I20260806 16:02:03.921082 9247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "548f7867ad1e4b3fae33485cc0241dbc"
format_stamp: "Formatted at 2026-08-06 16:02:03 on dist-test-slave-tm1g"
server_key: "f46fee9c724fd4d131d4de27b0de32a7"
server_key_iv: "2cc28f646ba2d23b36d279a0e4876f11"
server_key_version: "encryptionkey@0"
I20260806 16:02:03.924772 9247 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 16:02:03.927191 9262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:03.928468 9247 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 16:02:03.928620 9247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "548f7867ad1e4b3fae33485cc0241dbc"
format_stamp: "Formatted at 2026-08-06 16:02:03 on dist-test-slave-tm1g"
server_key: "f46fee9c724fd4d131d4de27b0de32a7"
server_key_iv: "2cc28f646ba2d23b36d279a0e4876f11"
server_key_version: "encryptionkey@0"
I20260806 16:02:03.928730 9247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:02:03.941641 9247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:02:03.944769 9247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:02:03.944973 9247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:02:03.951864 9247 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:37419
I20260806 16:02:03.952168 9315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:37419 every 8 connection(s)
I20260806 16:02:03.952791 9247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:02:03.955523 9316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:02:03.957741 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 9247
I20260806 16:02:03.957844 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:02:03.958153 25726 external_mini_cluster.cc:1428] Setting key de45c4b65865fefb1bfef40d9af4188d
I20260806 16:02:03.961340 9316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Bootstrap starting.
I20260806 16:02:03.963569 9316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Neither blocks nor log segments found. Creating new log.
I20260806 16:02:03.964272 9316 log.cc:824] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Log is configured to *not* fsync() on all Append() calls
I20260806 16:02:03.966101 9316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: No bootstrap required, opened a new log
Aug 06 16:02:03 dist-test-slave-tm1g krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:02:03.969448 9316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } }
I20260806 16:02:03.969691 9316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:02:03.969775 9316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 548f7867ad1e4b3fae33485cc0241dbc, State: Initialized, Role: FOLLOWER
I20260806 16:02:03.970264 9316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } }
I20260806 16:02:03.970463 9316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:02:03.970554 9316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:02:03.970655 9316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:02:03.971563 9316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } }
I20260806 16:02:03.971920 9316 leader_election.cc:302] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 548f7867ad1e4b3fae33485cc0241dbc; no voters:
I20260806 16:02:03.972254 9316 leader_election.cc:288] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:02:03.972324 9322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:02:03.972527 9322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 1 LEADER]: Becoming Leader. State: Replica: 548f7867ad1e4b3fae33485cc0241dbc, State: Running, Role: LEADER
I20260806 16:02:03.972635 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:03.959575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49136 (local address 127.25.31.190:37419)
0806 16:02:03.959984 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:03.959999 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:03.960034 (+ 35us) server_negotiation.cc:407] Connection header received
0806 16:02:03.960684 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:03.960700 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:03.961011 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:03.961340 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:02:03.962129 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:03.963149 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:03.964072 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:03.964590 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:03.964752 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:03.967247 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:02:03.967253 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:02:03.967264 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:02:03.967289 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:02:03.969432 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:03.969907 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:03.969913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:03.969919 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:03.970034 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:03.970351 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:03.970357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:03.970361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:03.970868 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:02:03.971068 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:03.971378 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 16:02:03.971661 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":123,"threads_started":1}
I20260806 16:02:03.973048 9322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } }
I20260806 16:02:03.973522 9316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:02:03.976119 9324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 548f7867ad1e4b3fae33485cc0241dbc. Latest consensus state: current_term: 1 leader_uuid: "548f7867ad1e4b3fae33485cc0241dbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } } }
I20260806 16:02:03.976156 9323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "548f7867ad1e4b3fae33485cc0241dbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } } }
I20260806 16:02:03.976279 9323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [sys.catalog]: This master's current role is: LEADER
I20260806 16:02:03.976279 9324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [sys.catalog]: This master's current role is: LEADER
W20260806 16:02:03.979108 9338 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:02:03.979171 9338 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:02:03.979246 9339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:02:03.979916 9339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:02:03.985186 9339 catalog_manager.cc:1347] Generated new cluster ID: 1cb2b5b9a9b644e599e9af3e7935c25c
I20260806 16:02:03.985494 9339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:02:04.008971 9339 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:02:04.010260 9339 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:02:04.019380 9339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Generated new TSK 0
I20260806 16:02:04.020213 9339 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 16:02:04.048943 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37419
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:02:04.159698 9345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:02:04.159930 9345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:02:04.159971 9345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:02:04.164548 9345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:02:04.164599 9345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:02:04.164701 9345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:02:04.169814 9345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:02:04.170881 9345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:02:04.171868 9345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:02:04.181644 9350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:04.181658 9351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:04.181885 9353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:04.181720 9345 server_base.cc:1034] running on GCE node
I20260806 16:02:04.182372 9345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:02:04.183001 9345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:02:04.184190 9345 hybrid_clock.cc:648] HybridClock initialized: now 1786032124184163 us; error 52 us; skew 500 ppm
Aug 06 16:02:04 dist-test-slave-tm1g krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032124, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:02:04.186993 9345 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 16:02:04.187963 9345 webserver.cc:460] Webserver started at http://127.25.31.129:33319/ using document root <none> and password file <none>
I20260806 16:02:04.188547 9345 fs_manager.cc:359] Metadata directory not provided
I20260806 16:02:04.188642 9345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:02:04.188886 9345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:02:04.190959 9345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "72f7109b3a1b4a5e95fb1b831f1350f2"
format_stamp: "Formatted at 2026-08-06 16:02:04 on dist-test-slave-tm1g"
server_key: "96d56ebfb26229bd0b714a74bb36a79d"
server_key_iv: "9412760441f61205eee36a8e11a3b078"
server_key_version: "encryptionkey@0"
I20260806 16:02:04.191540 9345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "72f7109b3a1b4a5e95fb1b831f1350f2"
format_stamp: "Formatted at 2026-08-06 16:02:04 on dist-test-slave-tm1g"
server_key: "96d56ebfb26229bd0b714a74bb36a79d"
server_key_iv: "9412760441f61205eee36a8e11a3b078"
server_key_version: "encryptionkey@0"
I20260806 16:02:04.195286 9345 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 16:02:04.197683 9360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:04.198961 9345 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 16:02:04.199100 9345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "72f7109b3a1b4a5e95fb1b831f1350f2"
format_stamp: "Formatted at 2026-08-06 16:02:04 on dist-test-slave-tm1g"
server_key: "96d56ebfb26229bd0b714a74bb36a79d"
server_key_iv: "9412760441f61205eee36a8e11a3b078"
server_key_version: "encryptionkey@0"
I20260806 16:02:04.199211 9345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:02:04.215507 9345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:02:04.218448 9345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:02:04.218648 9345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:02:04.219223 9345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:02:04.220032 9345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:02:04.220103 9345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:04.220187 9345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:02:04.220235 9345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:04.228901 9345 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:45833
I20260806 16:02:04.228925 9478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:45833 every 8 connection(s)
I20260806 16:02:04.229745 9345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 16:02:04.235146 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 9345
I20260806 16:02:04.235250 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 16:02:04.235631 25726 external_mini_cluster.cc:1428] Setting key bcff449598480397215b605e911c8db7
Aug 06 16:02:04 dist-test-slave-tm1g krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032124, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:02:04.244189 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.231683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:34619 (local address 127.25.31.190:37419)
0806 16:02:04.231837 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:04.231841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:04.232722 (+ 881us) server_negotiation.cc:407] Connection header received
0806 16:02:04.234117 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:04.234120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:04.234167 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:04.234245 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:02:04.235485 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.236211 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.237477 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.237716 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.237824 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.240565 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:02:04.240568 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:02:04.240571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:02:04.240599 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:02:04.242058 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:04.242638 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:04.242642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:04.242644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:04.242724 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:04.243099 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:04.243102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:04.243104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:04.243277 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:02:04.243383 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:04.243865 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 16:02:04.244013 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 16:02:04.244923 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.232069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:34619)
0806 16:02:04.232542 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:04.232585 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:04.233909 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:04.234433 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:04.234439 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:04.234779 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:02:04.235312 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.235337 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.236335 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.236338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.237342 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.237352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.237864 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.237868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.238040 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.238711 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:02:04.238738 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:02:04.240400 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:02:04.242180 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:04.242186 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:04.242201 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:04.242535 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:04.242815 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:04.242828 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:04.242831 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:04.242997 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:04.243390 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:02:04.243398 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:02:04.243632 (+ 234us) client_negotiation.cc:769] Sending connection context
0806 16:02:04.243807 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 16:02:04.244122 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
I20260806 16:02:04.246166 9479 heartbeater.cc:347] Connected to a master server at 127.25.31.190:37419
I20260806 16:02:04.246461 9479 heartbeater.cc:464] Registering TS with master...
I20260806 16:02:04.246966 9479 heartbeater.cc:511] Master 127.25.31.190:37419 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
I20260806 16:02:04.248514 9279 ts_manager.cc:194] Registered new tserver with Master: 72f7109b3a1b4a5e95fb1b831f1350f2 (127.25.31.129:45833)
I20260806 16:02:04.249995 9279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:34619
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 16:02:04.263648 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37419
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:02:04.371784 9485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:02:04.372053 9485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:02:04.372123 9485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:02:04.376237 9485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:02:04.376314 9485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:02:04.376430 9485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:02:04.381428 9485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:02:04.382462 9485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:02:04.383417 9485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:02:04.392720 9490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:04.392707 9491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:04.392709 9493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:04.393013 9485 server_base.cc:1034] running on GCE node
I20260806 16:02:04.393498 9485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:02:04.394059 9485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:02:04.395226 9485 hybrid_clock.cc:648] HybridClock initialized: now 1786032124395201 us; error 42 us; skew 500 ppm
Aug 06 16:02:04 dist-test-slave-tm1g krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032124, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:02:04.397806 9485 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 16:02:04.398718 9485 webserver.cc:460] Webserver started at http://127.25.31.130:41847/ using document root <none> and password file <none>
I20260806 16:02:04.399266 9485 fs_manager.cc:359] Metadata directory not provided
I20260806 16:02:04.399326 9485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:02:04.399498 9485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:02:04.401372 9485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75"
format_stamp: "Formatted at 2026-08-06 16:02:04 on dist-test-slave-tm1g"
server_key: "aca0ae525da91ce01cfd608195826026"
server_key_iv: "8f77b5b23f7b22926c26eee09bdaf212"
server_key_version: "encryptionkey@0"
I20260806 16:02:04.401870 9485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75"
format_stamp: "Formatted at 2026-08-06 16:02:04 on dist-test-slave-tm1g"
server_key: "aca0ae525da91ce01cfd608195826026"
server_key_iv: "8f77b5b23f7b22926c26eee09bdaf212"
server_key_version: "encryptionkey@0"
I20260806 16:02:04.405582 9485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 16:02:04.408026 9500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:04.409260 9485 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:02:04.409399 9485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75"
format_stamp: "Formatted at 2026-08-06 16:02:04 on dist-test-slave-tm1g"
server_key: "aca0ae525da91ce01cfd608195826026"
server_key_iv: "8f77b5b23f7b22926c26eee09bdaf212"
server_key_version: "encryptionkey@0"
I20260806 16:02:04.409509 9485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:02:04.425843 9485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:02:04.428931 9485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:02:04.429126 9485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:02:04.429736 9485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:02:04.430609 9485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:02:04.430681 9485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:04.430752 9485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:02:04.430804 9485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:04.439990 9485 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:45345
I20260806 16:02:04.440019 9618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:45345 every 8 connection(s)
I20260806 16:02:04.440821 9485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:02:04.449729 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 9485
I20260806 16:02:04.449864 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 16:02:04.450202 25726 external_mini_cluster.cc:1428] Setting key 868a8478778336ca36d74aabbfa84a0c
Aug 06 16:02:04 dist-test-slave-tm1g krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032124, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:02:04.455006 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.442622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:37725 (local address 127.25.31.190:37419)
0806 16:02:04.442773 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:04.442776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:04.443604 (+ 828us) server_negotiation.cc:407] Connection header received
0806 16:02:04.444960 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:04.444963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:04.445023 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:04.445135 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:02:04.446281 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.446877 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.448050 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.448290 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.448419 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.451144 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:02:04.451148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:02:04.451150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:02:04.451173 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:02:04.452642 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:04.453295 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:04.453301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:04.453305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:04.453447 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:04.453771 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:04.453776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:04.453780 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:04.454024 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:02:04.454131 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:04.454635 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 16:02:04.454773 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 16:02:04.455595 9621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.442974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:37725)
0806 16:02:04.443450 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:04.443494 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:04.444740 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:04.445319 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:04.445326 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:04.445665 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:02:04.446140 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.446153 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.447011 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.447020 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.447923 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.447933 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.448461 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.448466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.448631 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.449169 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:02:04.449184 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:02:04.450970 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:02:04.452765 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:04.452774 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:04.452790 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:04.453168 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:04.453547 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:04.453550 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:04.453551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:04.453676 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:04.454126 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:02:04.454130 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:02:04.454398 (+ 268us) client_negotiation.cc:769] Sending connection context
0806 16:02:04.454552 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 16:02:04.454785 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":115,"threads_started":1}
I20260806 16:02:04.456755 9619 heartbeater.cc:347] Connected to a master server at 127.25.31.190:37419
I20260806 16:02:04.456993 9619 heartbeater.cc:464] Registering TS with master...
I20260806 16:02:04.457449 9619 heartbeater.cc:511] Master 127.25.31.190:37419 requested a full tablet report, sending...
I20260806 16:02:04.458513 9279 ts_manager.cc:194] Registered new tserver with Master: 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345)
I20260806 16:02:04.459074 9279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:37725
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 16:02:04.477447 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37419
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:02:04.586460 9624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:02:04.586709 9624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:02:04.586789 9624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:02:04.590848 9624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:02:04.590904 9624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:02:04.591022 9624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:02:04.596204 9624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:02:04.597203 9624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:02:04.598186 9624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:02:04.607780 9629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:04.607770 9630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:04.607848 9624 server_base.cc:1034] running on GCE node
W20260806 16:02:04.607771 9632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:04.608445 9624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:02:04.608995 9624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:02:04.610157 9624 hybrid_clock.cc:648] HybridClock initialized: now 1786032124610122 us; error 46 us; skew 500 ppm
Aug 06 16:02:04 dist-test-slave-tm1g krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032124, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:02:04.612926 9624 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 16:02:04.613833 9624 webserver.cc:460] Webserver started at http://127.25.31.131:41939/ using document root <none> and password file <none>
I20260806 16:02:04.614441 9624 fs_manager.cc:359] Metadata directory not provided
I20260806 16:02:04.614506 9624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:02:04.614667 9624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:02:04.616550 9624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0"
format_stamp: "Formatted at 2026-08-06 16:02:04 on dist-test-slave-tm1g"
server_key: "e8a5155af0a746bfbb1b244b99b30d4a"
server_key_iv: "eb8ea19ce677a8755edf2879bfd728a7"
server_key_version: "encryptionkey@0"
I20260806 16:02:04.617063 9624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0"
format_stamp: "Formatted at 2026-08-06 16:02:04 on dist-test-slave-tm1g"
server_key: "e8a5155af0a746bfbb1b244b99b30d4a"
server_key_iv: "eb8ea19ce677a8755edf2879bfd728a7"
server_key_version: "encryptionkey@0"
I20260806 16:02:04.621101 9624 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 16:02:04.623481 9639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:04.624655 9624 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:02:04.624794 9624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0"
format_stamp: "Formatted at 2026-08-06 16:02:04 on dist-test-slave-tm1g"
server_key: "e8a5155af0a746bfbb1b244b99b30d4a"
server_key_iv: "eb8ea19ce677a8755edf2879bfd728a7"
server_key_version: "encryptionkey@0"
I20260806 16:02:04.624909 9624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:02:04.664285 9624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:02:04.667634 9624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:02:04.667842 9624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:02:04.668444 9624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:02:04.669269 9624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:02:04.669339 9624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:04.669414 9624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:02:04.669468 9624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:04.678651 9624 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:43571
I20260806 16:02:04.678679 9755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:43571 every 8 connection(s)
I20260806 16:02:04.679502 9624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:02:04.683738 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 9624
I20260806 16:02:04.683863 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 16:02:04.684211 25726 external_mini_cluster.cc:1428] Setting key c28f3f70da8d6c9591310e61b3992760
Aug 06 16:02:04 dist-test-slave-tm1g krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032124, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:02:04.693150 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.681281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:59399 (local address 127.25.31.190:37419)
0806 16:02:04.681443 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:04.681448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:04.682379 (+ 931us) server_negotiation.cc:407] Connection header received
0806 16:02:04.683848 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:04.683854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:04.683941 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:04.684039 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:02:04.685484 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.686079 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.687326 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.687559 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.687673 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.689979 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:02:04.689982 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:02:04.689984 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:02:04.690007 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:02:04.691308 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:04.691900 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:04.691903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:04.691904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:04.691974 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:04.692253 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:04.692255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:04.692256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:04.692434 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:02:04.692509 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:04.692940 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 16:02:04.693032 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 16:02:04.693918 9758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.681756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:59399)
0806 16:02:04.682217 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:04.682262 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:04.683565 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:04.684252 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:04.684262 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:04.684691 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:02:04.685280 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.685317 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.686220 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.686223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.687194 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.687204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.687714 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.687718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.687907 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.688429 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:02:04.688444 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:02:04.689825 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:02:04.691439 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:04.691443 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:04.691453 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:04.691808 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:04.692051 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:04.692053 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:04.692054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:04.692179 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:04.692521 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:02:04.692526 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:02:04.692735 (+ 209us) client_negotiation.cc:769] Sending connection context
0806 16:02:04.692892 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 16:02:04.693126 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":124,"threads_started":1}
I20260806 16:02:04.695040 9756 heartbeater.cc:347] Connected to a master server at 127.25.31.190:37419
I20260806 16:02:04.695242 9756 heartbeater.cc:464] Registering TS with master...
I20260806 16:02:04.695647 9756 heartbeater.cc:511] Master 127.25.31.190:37419 requested a full tablet report, sending...
I20260806 16:02:04.696630 9279 ts_manager.cc:194] Registered new tserver with Master: b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571)
I20260806 16:02:04.697391 9279 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:59399
I20260806 16:02:04.698153 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:02:04.707396 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.700393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49148 (local address 127.25.31.190:37419)
0806 16:02:04.700506 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:04.700509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:04.700657 (+ 148us) server_negotiation.cc:407] Connection header received
0806 16:02:04.700794 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:04.700797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:04.700835 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:04.700916 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:02:04.701593 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.702121 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.702964 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.703201 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.703293 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.704481 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:02:04.704484 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:02:04.704486 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:02:04.704506 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:02:04.705931 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:04.706286 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:04.706288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:04.706289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:04.706417 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:04.706650 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:04.706653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:04.706654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:04.706844 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:02:04.706959 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:04.707173 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 16:02:04.707258 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 16:02:04.710160 9279 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:02:04.713300 9279 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:02:04.727762 9768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.722335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:45345 (local address 127.0.0.1:53090)
0806 16:02:04.723018 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:04.723031 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:04.723125 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:04.723829 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:04.723832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:04.723847 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:02:04.723925 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.723929 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.724919 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.724922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.726277 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.726286 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.727211 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.727215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.727322 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.727362 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 16:02:04.727474 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 16:02:04.727598 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":115,"thread_start_us":129,"threads_started":1}
I20260806 16:02:04.727797 9772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.722494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53090 (local address 127.25.31.130:45345)
0806 16:02:04.722882 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:04.722895 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:04.723223 (+ 328us) server_negotiation.cc:407] Connection header received
0806 16:02:04.723310 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:04.723316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:04.723519 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:04.723662 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:02:04.724038 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.724780 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.726431 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.727083 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.727191 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.727333 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:04.727518 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 16:02:04.727622 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":96,"threads_started":1}
I20260806 16:02:04.728698 9767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.721593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:45833 (local address 127.0.0.1:50884)
0806 16:02:04.721939 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:04.721995 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:04.722117 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:04.723512 (+ 1395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:04.723518 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:04.723541 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:02:04.723711 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.723720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.725229 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.725232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.726826 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.726838 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.728180 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.728187 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.728318 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.728355 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 16:02:04.728436 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 16:02:04.728527 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":193,"spinlock_wait_cycles":4992,"thread_start_us":110,"threads_started":1}
I20260806 16:02:04.728848 9771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.722430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36752 (local address 127.25.31.131:43571)
0806 16:02:04.722882 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:04.722895 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:04.722919 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:02:04.723002 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:04.723008 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:04.723210 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:04.723365 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:02:04.723885 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.725089 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.726971 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.727835 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.727947 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.728082 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:04.728612 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 16:02:04.728715 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":141,"threads_started":1}
I20260806 16:02:04.729311 9770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.721685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:43571 (local address 127.0.0.1:36752)
0806 16:02:04.722579 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:04.722593 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:04.722685 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:04.723569 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:04.723573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:04.723605 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:02:04.723724 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.723730 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.725274 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.725277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.726826 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.726838 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.728151 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.728154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.728283 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.728321 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 16:02:04.728411 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 16:02:04.728502 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":160,"spinlock_wait_cycles":3072,"thread_start_us":111,"threads_started":1}
I20260806 16:02:04.729745 9769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.721703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50884 (local address 127.25.31.129:45833)
0806 16:02:04.723090 (+ 1387us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:04.723104 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:04.723124 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:02:04.723194 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:04.723198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:04.723356 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:04.723493 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:02:04.723851 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.725089 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.726981 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.727865 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.727988 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.728118 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:04.729499 (+ 1381us) server_negotiation.cc:299] Negotiation successful
0806 16:02:04.729601 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1290,"thread_start_us":93,"threads_started":1}
I20260806 16:02:04.731546 9548 tablet_service.cc:1467] Processing CreateTablet for tablet 43bd433874894cc49dac6184a2f0e726 (DEFAULT_TABLE table=test-table [id=af30bf1f4bdd4d70a89a0e1b29c12874]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:02:04.732764 9548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43bd433874894cc49dac6184a2f0e726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:02:04.732529 9410 tablet_service.cc:1467] Processing CreateTablet for tablet 43bd433874894cc49dac6184a2f0e726 (DEFAULT_TABLE table=test-table [id=af30bf1f4bdd4d70a89a0e1b29c12874]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:02:04.732693 9688 tablet_service.cc:1467] Processing CreateTablet for tablet 43bd433874894cc49dac6184a2f0e726 (DEFAULT_TABLE table=test-table [id=af30bf1f4bdd4d70a89a0e1b29c12874]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:02:04.733526 9410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43bd433874894cc49dac6184a2f0e726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:02:04.733664 9688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43bd433874894cc49dac6184a2f0e726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:02:04.739506 9773 tablet_bootstrap.cc:492] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0: Bootstrap starting.
I20260806 16:02:04.740778 9774 tablet_bootstrap.cc:492] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: Bootstrap starting.
I20260806 16:02:04.742501 9774 tablet_bootstrap.cc:654] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: Neither blocks nor log segments found. Creating new log.
I20260806 16:02:04.742544 9773 tablet_bootstrap.cc:654] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0: Neither blocks nor log segments found. Creating new log.
I20260806 16:02:04.742653 9775 tablet_bootstrap.cc:492] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Bootstrap starting.
I20260806 16:02:04.743839 9773 log.cc:824] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0: Log is configured to *not* fsync() on all Append() calls
I20260806 16:02:04.743844 9774 log.cc:824] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: Log is configured to *not* fsync() on all Append() calls
I20260806 16:02:04.745093 9775 tablet_bootstrap.cc:654] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Neither blocks nor log segments found. Creating new log.
I20260806 16:02:04.745844 9774 tablet_bootstrap.cc:492] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: No bootstrap required, opened a new log
I20260806 16:02:04.745851 9773 tablet_bootstrap.cc:492] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0: No bootstrap required, opened a new log
I20260806 16:02:04.745834 9775 log.cc:824] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Log is configured to *not* fsync() on all Append() calls
I20260806 16:02:04.746040 9774 ts_tablet_manager.cc:1397] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 16:02:04.746040 9773 ts_tablet_manager.cc:1397] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 16:02:04.747442 9775 tablet_bootstrap.cc:492] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: No bootstrap required, opened a new log
I20260806 16:02:04.747668 9775 ts_tablet_manager.cc:1397] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 16:02:04.749879 9774 raft_consensus.cc:348] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } } peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } }
I20260806 16:02:04.750156 9774 raft_consensus.cc:374] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:02:04.750250 9774 raft_consensus.cc:729] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72f7109b3a1b4a5e95fb1b831f1350f2, State: Initialized, Role: FOLLOWER
I20260806 16:02:04.749879 9773 raft_consensus.cc:348] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } } peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } }
I20260806 16:02:04.750591 9773 raft_consensus.cc:374] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:02:04.750666 9773 raft_consensus.cc:729] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4b44a0d86f64554b32e15d1b4fa8ba0, State: Initialized, Role: FOLLOWER
I20260806 16:02:04.750636 9775 raft_consensus.cc:348] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } } peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } }
I20260806 16:02:04.750787 9774 consensus_queue.cc:260] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } } peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } }
I20260806 16:02:04.751031 9775 raft_consensus.cc:374] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:02:04.751149 9775 raft_consensus.cc:729] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82ecd0aa5a2e4a428ba6d37b4a0e1a75, State: Initialized, Role: FOLLOWER
I20260806 16:02:04.751084 9773 consensus_queue.cc:260] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } } peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } }
I20260806 16:02:04.751447 9479 heartbeater.cc:503] Master 127.25.31.190:37419 was elected leader, sending a full tablet report...
I20260806 16:02:04.751621 9775 consensus_queue.cc:260] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } } peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } }
I20260806 16:02:04.751747 9774 ts_tablet_manager.cc:1428] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260806 16:02:04.751932 9773 ts_tablet_manager.cc:1428] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 16:02:04.752240 9756 heartbeater.cc:503] Master 127.25.31.190:37419 was elected leader, sending a full tablet report...
I20260806 16:02:04.752492 9775 ts_tablet_manager.cc:1428] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 16:02:04.752810 9619 heartbeater.cc:503] Master 127.25.31.190:37419 was elected leader, sending a full tablet report...
I20260806 16:02:04.763111 9781 raft_consensus.cc:482] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:02:04.763245 9781 raft_consensus.cc:504] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } } peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } }
I20260806 16:02:04.764307 9781 leader_election.cc:288] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72f7109b3a1b4a5e95fb1b831f1350f2 (127.25.31.129:45833), b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571)
I20260806 16:02:04.767920 9621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.764477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:45833 (local address 127.25.31.130:47747)
0806 16:02:04.764682 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:04.764698 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:04.764806 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:04.765049 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:04.765051 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:04.765069 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:02:04.765188 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.765194 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.765962 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.765967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.766851 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.766861 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.767498 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.767501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.767613 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.767631 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 16:02:04.767720 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 16:02:04.767790 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260806 16:02:04.767918 9769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.764568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:47747 (local address 127.25.31.129:45833)
0806 16:02:04.764705 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:04.764710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:04.764802 (+ 92us) server_negotiation.cc:407] Connection header received
0806 16:02:04.764900 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:04.764903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:04.764952 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:04.765038 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:02:04.765308 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.765848 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.766978 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.767370 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.767500 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.767552 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:04.767737 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 16:02:04.767805 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 16:02:04.768085 9771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.764637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:54041 (local address 127.25.31.131:43571)
0806 16:02:04.764757 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:04.764761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:04.764992 (+ 231us) server_negotiation.cc:407] Connection header received
0806 16:02:04.765104 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:04.765108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:04.765155 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:04.765238 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:02:04.765522 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.766019 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.767118 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.767513 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.767641 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.767691 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:04.767893 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 16:02:04.767961 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 16:02:04.768146 9782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.764558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:43571 (local address 127.25.31.130:54041)
0806 16:02:04.764886 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:04.764897 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:04.765018 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:04.765249 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:04.765252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:04.765268 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:02:04.765400 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.765406 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.766133 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.766137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.766993 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:04.767001 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.767641 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.767645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:04.767753 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.767774 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 16:02:04.767863 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 16:02:04.767935 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":114,"threads_started":1}
I20260806 16:02:04.768596 9431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43bd433874894cc49dac6184a2f0e726" candidate_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" is_pre_election: true
I20260806 16:02:04.768765 9708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43bd433874894cc49dac6184a2f0e726" candidate_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" is_pre_election: true
I20260806 16:02:04.768900 9431 raft_consensus.cc:2454] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82ecd0aa5a2e4a428ba6d37b4a0e1a75 in term 0.
I20260806 16:02:04.769001 9708 raft_consensus.cc:2454] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82ecd0aa5a2e4a428ba6d37b4a0e1a75 in term 0.
I20260806 16:02:04.769418 9504 leader_election.cc:302] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72f7109b3a1b4a5e95fb1b831f1350f2, 82ecd0aa5a2e4a428ba6d37b4a0e1a75; no voters:
I20260806 16:02:04.769691 9781 raft_consensus.cc:2781] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:02:04.769776 9781 raft_consensus.cc:482] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:02:04.769819 9781 raft_consensus.cc:3037] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:02:04.770918 9781 raft_consensus.cc:504] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } } peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } }
I20260806 16:02:04.771255 9781 leader_election.cc:288] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [CANDIDATE]: Term 1 election: Requested vote from peers 72f7109b3a1b4a5e95fb1b831f1350f2 (127.25.31.129:45833), b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571)
I20260806 16:02:04.771567 9431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43bd433874894cc49dac6184a2f0e726" candidate_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2"
I20260806 16:02:04.771708 9708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43bd433874894cc49dac6184a2f0e726" candidate_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0"
I20260806 16:02:04.771685 9431 raft_consensus.cc:3037] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:02:04.771868 9708 raft_consensus.cc:3037] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:02:04.772795 9431 raft_consensus.cc:2454] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82ecd0aa5a2e4a428ba6d37b4a0e1a75 in term 1.
I20260806 16:02:04.772969 9708 raft_consensus.cc:2454] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82ecd0aa5a2e4a428ba6d37b4a0e1a75 in term 1.
I20260806 16:02:04.773145 9504 leader_election.cc:302] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72f7109b3a1b4a5e95fb1b831f1350f2, 82ecd0aa5a2e4a428ba6d37b4a0e1a75; no voters:
I20260806 16:02:04.773373 9781 raft_consensus.cc:2781] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:02:04.773573 9781 raft_consensus.cc:686] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 1 LEADER]: Becoming Leader. State: Replica: 82ecd0aa5a2e4a428ba6d37b4a0e1a75, State: Running, Role: LEADER
I20260806 16:02:04.773880 9781 consensus_queue.cc:237] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } } peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } }
I20260806 16:02:04.777034 9279 catalog_manager.cc:5495] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130). New cstate: current_term: 1 leader_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } health_report { overall_health: HEALTHY } } }
I20260806 16:02:04.789276 9772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.785740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53106 (local address 127.25.31.130:45345)
0806 16:02:04.785876 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:04.785880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:04.785894 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:02:04.786049 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:04.786052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:04.786102 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:04.786209 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:02:04.786501 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.787002 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.787850 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:04.788070 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:04.788181 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:04.788491 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:02:04.788870 (+ 379us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:02:04.788998 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:04.789066 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 16:02:04.789122 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 16:02:04.794770 9431 raft_consensus.cc:1261] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Refusing update from remote peer 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:02:04.794770 9708 raft_consensus.cc:1261] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 1 FOLLOWER]: Refusing update from remote peer 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:02:04.795440 9781 consensus_queue.cc:1035] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:02:04.795660 9784 consensus_queue.cc:1035] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:02:04.803642 9789 mvcc.cc:205] Tried to move back new op lower bound from 7315587583152390144 to 7315587583076913152. Current Snapshot: MvccSnapshot[applied={T|T < 7315587583152390144}]
I20260806 16:02:04.813902 9278 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:02:04.814133 9278 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:02:04.816880 9278 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Sending DeleteTablet for 3 replicas of tablet 43bd433874894cc49dac6184a2f0e726
I20260806 16:02:04.817762 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 9345
I20260806 16:02:04.817747 9688 tablet_service.cc:1514] Processing DeleteTablet for tablet 43bd433874894cc49dac6184a2f0e726 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:02:04 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:36752
I20260806 16:02:04.817799 9410 tablet_service.cc:1514] Processing DeleteTablet for tablet 43bd433874894cc49dac6184a2f0e726 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:02:04 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:50884
I20260806 16:02:04.819334 9548 tablet_service.cc:1514] Processing DeleteTablet for tablet 43bd433874894cc49dac6184a2f0e726 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:02:04 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:53090
I20260806 16:02:04.819818 9799 tablet_replica.cc:330] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: stopping tablet replica
I20260806 16:02:04.819921 9798 tablet_replica.cc:330] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0: stopping tablet replica
I20260806 16:02:04.820179 9799 raft_consensus.cc:2229] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:02:04.820286 9798 raft_consensus.cc:2229] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:02:04.820655 9799 raft_consensus.cc:2258] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:02:04.820674 9798 raft_consensus.cc:2258] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:02:04.822250 9798 ts_tablet_manager.cc:1905] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:02:04.822620 9799 ts_tablet_manager.cc:1905] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:02:04.825102 9798 ts_tablet_manager.cc:1918] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:02:04.825271 9798 log.cc:1196] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/43bd433874894cc49dac6184a2f0e726
I20260806 16:02:04.825702 9798 ts_tablet_manager.cc:1939] T 43bd433874894cc49dac6184a2f0e726 P b4b44a0d86f64554b32e15d1b4fa8ba0: Deleting consensus metadata
I20260806 16:02:04.826120 9799 ts_tablet_manager.cc:1918] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:02:04.826287 9799 log.cc:1196] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/43bd433874894cc49dac6184a2f0e726
I20260806 16:02:04.826730 9799 ts_tablet_manager.cc:1939] T 43bd433874894cc49dac6184a2f0e726 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Deleting consensus metadata
I20260806 16:02:04.827101 9265 catalog_manager.cc:4841] TS b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571): tablet 43bd433874894cc49dac6184a2f0e726 (table test-table [id=af30bf1f4bdd4d70a89a0e1b29c12874]) successfully deleted
W20260806 16:02:04.827422 9266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:45833 (error 108)
I20260806 16:02:04.827909 9266 catalog_manager.cc:4841] TS 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345): tablet 43bd433874894cc49dac6184a2f0e726 (table test-table [id=af30bf1f4bdd4d70a89a0e1b29c12874]) successfully deleted
I20260806 16:02:04.828433 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 9485
I20260806 16:02:04.829445 9770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:04.828745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:45833 (local address 127.0.0.1:50892)
0806 16:02:04.829296 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:04.829365 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458}
W20260806 16:02:04.829923 9266 catalog_manager.cc:4584] TS 72f7109b3a1b4a5e95fb1b831f1350f2 (127.25.31.129:45833): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 43bd433874894cc49dac6184a2f0e726: Network error: Client connection negotiation failed: client connection to 127.25.31.129:45833: connect: Connection refused (error 111)
I20260806 16:02:04.835341 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 9624
I20260806 16:02:04.841744 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 9247
I20260806 16:02:24.848841 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:37419
--webserver_interface=127.25.31.190
--webserver_port=35281
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:37419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:02:24.960176 9801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:02:24.960518 9801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:02:24.960606 9801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:02:24.964831 9801 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 16:02:24.964916 9801 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 16:02:24.964946 9801 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 16:02:24.964979 9801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:02:24.965023 9801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:02:24.965067 9801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:02:24.965106 9801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:02:24.965142 9801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:02:24.970434 9801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:37419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:37419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=35281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:02:24.971487 9801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:02:24.972515 9801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:02:24.985605 9807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:24.985595 9808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:24.985911 9801 server_base.cc:1034] running on GCE node
W20260806 16:02:24.985590 9810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:24.986624 9801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:02:24.987804 9801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:02:24.988965 9801 hybrid_clock.cc:648] HybridClock initialized: now 1786032144988934 us; error 45 us; skew 500 ppm
Aug 06 16:02:24 dist-test-slave-tm1g krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032144, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:02:24.996546 9801 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:02:24.997582 9801 webserver.cc:460] Webserver started at http://127.25.31.190:35281/ using document root <none> and password file <none>
I20260806 16:02:24.998241 9801 fs_manager.cc:359] Metadata directory not provided
I20260806 16:02:24.998400 9801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:02:25.002570 9801 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 16:02:25.004971 9817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:25.006462 9801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:02:25.006613 9801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "548f7867ad1e4b3fae33485cc0241dbc"
format_stamp: "Formatted at 2026-08-06 16:02:03 on dist-test-slave-tm1g"
server_key: "f46fee9c724fd4d131d4de27b0de32a7"
server_key_iv: "2cc28f646ba2d23b36d279a0e4876f11"
server_key_version: "encryptionkey@0"
I20260806 16:02:25.007273 9801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:02:25.019104 9801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:02:25.022414 9801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:02:25.022627 9801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:02:25.030043 9801 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:37419
I20260806 16:02:25.030387 9870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:37419 every 8 connection(s)
I20260806 16:02:25.031201 9801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:02:25.035662 9871 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:02:25.035852 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 9801
I20260806 16:02:25.036273 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:45833
--local_ip_for_outbound_sockets=127.25.31.129
--tserver_master_addrs=127.25.31.190:37419
--webserver_port=33319
--webserver_interface=127.25.31.129
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:02:25.037456 9871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Bootstrap starting.
I20260806 16:02:25.055364 9871 log.cc:824] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Log is configured to *not* fsync() on all Append() calls
I20260806 16:02:25.064476 9871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:02:25.064872 9871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Bootstrap complete.
I20260806 16:02:25.069015 9871 raft_consensus.cc:348] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } }
I20260806 16:02:25.069432 9871 raft_consensus.cc:729] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 548f7867ad1e4b3fae33485cc0241dbc, State: Initialized, Role: FOLLOWER
I20260806 16:02:25.069900 9871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } }
I20260806 16:02:25.070050 9871 raft_consensus.cc:388] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:02:25.070142 9871 raft_consensus.cc:482] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:02:25.070264 9871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 1 FOLLOWER]: Advancing to term 2
I20260806 16:02:25.072139 9871 raft_consensus.cc:504] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } }
I20260806 16:02:25.072484 9871 leader_election.cc:302] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 548f7867ad1e4b3fae33485cc0241dbc; no voters:
I20260806 16:02:25.072793 9871 leader_election.cc:288] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 16:02:25.072942 9875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 2 FOLLOWER]: Leader election won for term 2
I20260806 16:02:25.073174 9875 raft_consensus.cc:686] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 2 LEADER]: Becoming Leader. State: Replica: 548f7867ad1e4b3fae33485cc0241dbc, State: Running, Role: LEADER
I20260806 16:02:25.073539 9875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } }
I20260806 16:02:25.074018 9871 sys_catalog.cc:564] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:02:25.074970 9876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "548f7867ad1e4b3fae33485cc0241dbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } } }
I20260806 16:02:25.075096 9876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [sys.catalog]: This master's current role is: LEADER
I20260806 16:02:25.075430 9877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 548f7867ad1e4b3fae33485cc0241dbc. Latest consensus state: current_term: 2 leader_uuid: "548f7867ad1e4b3fae33485cc0241dbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } } }
I20260806 16:02:25.075563 9877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [sys.catalog]: This master's current role is: LEADER
I20260806 16:02:25.075871 9884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:02:25.080675 9884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:02:25.081444 9884 catalog_manager.cc:1259] Loaded cluster ID: 1cb2b5b9a9b644e599e9af3e7935c25c
I20260806 16:02:25.081521 9884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:02:25.084928 9884 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:02:25.085609 9884 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Loaded TSK: 0
I20260806 16:02:25.098758 9884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Generated new TSK 1
I20260806 16:02:25.099012 9884 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:02:25.155723 9873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:02:25.156004 9873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:02:25.156081 9873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:02:25.160076 9873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:02:25.160159 9873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:02:25.160244 9873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:02:25.165608 9873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:45833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=33319
--tserver_master_addrs=127.25.31.190:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:02:25.166707 9873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:02:25.167770 9873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:02:25.178532 9899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:25.178540 9900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:25.178741 9873 server_base.cc:1034] running on GCE node
W20260806 16:02:25.178588 9902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:25.179342 9873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:02:25.179970 9873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:02:25.181173 9873 hybrid_clock.cc:648] HybridClock initialized: now 1786032145181158 us; error 36 us; skew 500 ppm
Aug 06 16:02:25 dist-test-slave-tm1g krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032145, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:02:25.184159 9873 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 16:02:25.185073 9873 webserver.cc:460] Webserver started at http://127.25.31.129:33319/ using document root <none> and password file <none>
I20260806 16:02:25.185653 9873 fs_manager.cc:359] Metadata directory not provided
I20260806 16:02:25.185771 9873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:02:25.189815 9873 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:02:25.192217 9909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:25.193599 9873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:02:25.193735 9873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "72f7109b3a1b4a5e95fb1b831f1350f2"
format_stamp: "Formatted at 2026-08-06 16:02:04 on dist-test-slave-tm1g"
server_key: "96d56ebfb26229bd0b714a74bb36a79d"
server_key_iv: "9412760441f61205eee36a8e11a3b078"
server_key_version: "encryptionkey@0"
I20260806 16:02:25.194298 9873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:02:25.218794 9873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:02:25.222158 9873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:02:25.222409 9873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:02:25.223094 9873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:02:25.224560 9916 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 16:02:25.228716 9873 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 16:02:25.228786 9873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260806 16:02:25.228886 9873 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 16:02:25.230569 9873 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 16:02:25.230616 9873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260806 16:02:25.230890 9916 tablet_bootstrap.cc:492] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: Bootstrap starting.
I20260806 16:02:25.241192 9873 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:45833
I20260806 16:02:25.241245 10026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:45833 every 8 connection(s)
I20260806 16:02:25.242120 9873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 16:02:25.243409 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 9873
I20260806 16:02:25.243908 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:45345
--local_ip_for_outbound_sockets=127.25.31.130
--tserver_master_addrs=127.25.31.190:37419
--webserver_port=41847
--webserver_interface=127.25.31.130
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:02:25.251049 9916 log.cc:824] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:02:25 dist-test-slave-tm1g krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032145, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:02:25.260227 9916 tablet_bootstrap.cc:492] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:02:25.260715 9916 tablet_bootstrap.cc:492] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: Bootstrap complete.
I20260806 16:02:25.261047 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.244670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:49241 (local address 127.25.31.190:37419)
0806 16:02:25.245164 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:25.245183 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:25.245689 (+ 506us) server_negotiation.cc:407] Connection header received
0806 16:02:25.247297 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:25.247314 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:25.247711 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:25.248084 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:02:25.249659 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.251503 (+ 1844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.252798 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.253288 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.253474 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.255987 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:02:25.255992 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:02:25.256003 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:02:25.256033 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:02:25.257773 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:25.258355 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:25.258360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:25.258364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:25.258491 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:25.258895 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:25.258901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:25.258904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:25.259242 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:02:25.259390 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:25.259911 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 16:02:25.260174 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":115,"threads_started":1}
I20260806 16:02:25.261062 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.244414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:49241)
0806 16:02:25.245483 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:25.245548 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:25.246816 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:25.248188 (+ 1372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:25.248197 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:25.248757 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:02:25.249444 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.249464 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.251682 (+ 2218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.251686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.252661 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.252671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.253437 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.253441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.253658 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.254257 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:02:25.254273 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:02:25.255778 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:02:25.257904 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:25.257910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:25.257923 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:25.258194 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:25.258652 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:25.258655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:25.258657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:25.258792 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:25.259363 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:02:25.259369 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:02:25.259614 (+ 245us) client_negotiation.cc:769] Sending connection context
0806 16:02:25.259799 (+ 185us) client_negotiation.cc:240] Negotiation successful
0806 16:02:25.260145 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":152,"threads_started":1}
I20260806 16:02:25.262259 9916 ts_tablet_manager.cc:1397] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: Time spent bootstrapping tablet: real 0.032s user 0.025s sys 0.004s
I20260806 16:02:25.263728 10027 heartbeater.cc:347] Connected to a master server at 127.25.31.190:37419
I20260806 16:02:25.263952 10027 heartbeater.cc:464] Registering TS with master...
I20260806 16:02:25.264424 10027 heartbeater.cc:511] Master 127.25.31.190:37419 requested a full tablet report, sending...
I20260806 16:02:25.265365 9916 raft_consensus.cc:348] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } } peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } }
I20260806 16:02:25.265918 9916 raft_consensus.cc:729] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72f7109b3a1b4a5e95fb1b831f1350f2, State: Initialized, Role: FOLLOWER
I20260806 16:02:25.266438 9916 consensus_queue.cc:260] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } } peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } }
I20260806 16:02:25.266919 9834 ts_manager.cc:194] Registered new tserver with Master: 72f7109b3a1b4a5e95fb1b831f1350f2 (127.25.31.129:45833)
I20260806 16:02:25.267450 9916 ts_tablet_manager.cc:1428] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 16:02:25.270227 9834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:49241
I20260806 16:02:25.272595 10027 heartbeater.cc:503] Master 127.25.31.190:37419 was elected leader, sending a full tablet report...
Aug 06 16:02:25 dist-test-slave-tm1g krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032144, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for kudu/127.25.31.129@KRBTEST.COM
I20260806 16:02:25.280783 10034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.269718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:45833 (local address 127.0.0.1:43552)
0806 16:02:25.270406 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:25.270449 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:25.270584 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:25.271234 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:25.271240 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:25.271666 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:02:25.271797 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.271805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.273090 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.273093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.274080 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.274092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.274869 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.274872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.274956 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.275563 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:02:25.275575 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:02:25.276974 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:02:25.279070 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:25.279078 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:25.279082 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:25.279300 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:25.279627 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:25.279630 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:25.279631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:25.279734 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:25.280251 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:02:25.280256 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:02:25.280366 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 16:02:25.280484 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 16:02:25.280636 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":126,"threads_started":1}
I20260806 16:02:25.280920 10035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.269793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43552 (local address 127.25.31.129:45833)
0806 16:02:25.270135 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:25.270146 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:25.270557 (+ 411us) server_negotiation.cc:407] Connection header received
0806 16:02:25.270735 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:25.270740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:25.270940 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:25.271072 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:02:25.271938 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.272955 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.274216 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.274762 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.274879 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.277112 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:02:25.277123 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:02:25.277135 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:02:25.277162 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:02:25.278966 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:25.279410 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:25.279416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:25.279418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:25.279512 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:25.279832 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:25.279835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:25.279837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:25.280126 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:02:25.280254 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:25.280587 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 16:02:25.280754 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":91,"threads_started":1}
I20260806 16:02:25.281926 9960 tablet_service.cc:1514] Processing DeleteTablet for tablet 43bd433874894cc49dac6184a2f0e726 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:02:04 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:43552
I20260806 16:02:25.282939 10036 tablet_replica.cc:330] stopping tablet replica
I20260806 16:02:25.283241 10036 raft_consensus.cc:2229] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:02:25.283499 10036 raft_consensus.cc:2258] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:02:25.284462 10036 ts_tablet_manager.cc:1905] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:02:25.288219 10036 ts_tablet_manager.cc:1918] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:02:25.288301 10036 log.cc:1196] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal/wals/43bd433874894cc49dac6184a2f0e726
I20260806 16:02:25.288614 10036 ts_tablet_manager.cc:1939] T 43bd433874894cc49dac6184a2f0e726 P 72f7109b3a1b4a5e95fb1b831f1350f2: Deleting consensus metadata
I20260806 16:02:25.290275 9821 catalog_manager.cc:4841] TS 72f7109b3a1b4a5e95fb1b831f1350f2 (127.25.31.129:45833): tablet 43bd433874894cc49dac6184a2f0e726 (table test-table [id=af30bf1f4bdd4d70a89a0e1b29c12874]) successfully deleted
W20260806 16:02:25.366425 10031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:02:25.366670 10031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:02:25.366710 10031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:02:25.370585 10031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:02:25.370633 10031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:02:25.370687 10031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:02:25.376117 10031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:45345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=41847
--tserver_master_addrs=127.25.31.190:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:02:25.377018 10031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:02:25.377938 10031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:02:25.387231 10041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:25.387252 10042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:25.387270 10044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:25.387432 10031 server_base.cc:1034] running on GCE node
I20260806 16:02:25.387814 10031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:02:25.388447 10031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:02:25.389638 10031 hybrid_clock.cc:648] HybridClock initialized: now 1786032145389609 us; error 46 us; skew 500 ppm
Aug 06 16:02:25 dist-test-slave-tm1g krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032145, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:02:25.392364 10031 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 16:02:25.393178 10031 webserver.cc:460] Webserver started at http://127.25.31.130:41847/ using document root <none> and password file <none>
I20260806 16:02:25.393847 10031 fs_manager.cc:359] Metadata directory not provided
I20260806 16:02:25.393958 10031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:02:25.397740 10031 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:02:25.399951 10051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:25.401137 10031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:02:25.401269 10031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75"
format_stamp: "Formatted at 2026-08-06 16:02:04 on dist-test-slave-tm1g"
server_key: "aca0ae525da91ce01cfd608195826026"
server_key_iv: "8f77b5b23f7b22926c26eee09bdaf212"
server_key_version: "encryptionkey@0"
I20260806 16:02:25.401815 10031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:02:25.413583 10031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:02:25.416720 10031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:02:25.416915 10031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:02:25.417534 10031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:02:25.418424 10031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:02:25.418511 10031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:25.418581 10031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:02:25.418634 10031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:25.427309 10031 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:45345
I20260806 16:02:25.427338 10169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:45345 every 8 connection(s)
I20260806 16:02:25.428258 10031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:02:25.431157 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 10031
I20260806 16:02:25.431638 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:43571
--local_ip_for_outbound_sockets=127.25.31.131
--tserver_master_addrs=127.25.31.190:37419
--webserver_port=41939
--webserver_interface=127.25.31.131
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 16:02:25 dist-test-slave-tm1g krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032145, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:02:25.443356 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.429981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:47589 (local address 127.25.31.190:37419)
0806 16:02:25.430119 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:25.430122 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:25.431273 (+ 1151us) server_negotiation.cc:407] Connection header received
0806 16:02:25.432540 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:25.432542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:25.432595 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:25.432667 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:02:25.433997 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.434909 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.436118 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.436568 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.436705 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.439507 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:02:25.439510 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:02:25.439512 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:02:25.439537 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:02:25.441003 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:25.441655 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:25.441658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:25.441659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:25.441733 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:25.442148 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:25.442150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:25.442151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:25.442372 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:02:25.442451 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:25.443120 (+ 669us) server_negotiation.cc:299] Negotiation successful
0806 16:02:25.443231 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 16:02:25.444397 10172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.430402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:47589)
0806 16:02:25.431071 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:25.431129 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:25.432316 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:25.432836 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:25.432847 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:25.433366 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:02:25.433838 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.433851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.435033 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.435036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.435979 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.435990 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.436719 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.436721 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.436855 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.437428 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:02:25.437443 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:02:25.439326 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:02:25.441131 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:25.441139 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:25.441157 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:25.441554 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:25.441829 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:25.441835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:25.441839 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:25.442046 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:25.442485 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:02:25.442494 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:02:25.442827 (+ 333us) client_negotiation.cc:769] Sending connection context
0806 16:02:25.443078 (+ 251us) client_negotiation.cc:240] Negotiation successful
0806 16:02:25.443440 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":102,"threads_started":1}
I20260806 16:02:25.445564 10170 heartbeater.cc:347] Connected to a master server at 127.25.31.190:37419
I20260806 16:02:25.445801 10170 heartbeater.cc:464] Registering TS with master...
I20260806 16:02:25.446295 10170 heartbeater.cc:511] Master 127.25.31.190:37419 requested a full tablet report, sending...
I20260806 16:02:25.447393 9834 ts_manager.cc:194] Registered new tserver with Master: 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345)
I20260806 16:02:25.447965 9834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:47589
W20260806 16:02:25.543736 10173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:02:25.544018 10173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:02:25.544101 10173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:02:25.548156 10173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:02:25.548237 10173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:02:25.548329 10173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:02:25.553366 10173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:43571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=41939
--tserver_master_addrs=127.25.31.190:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:02:25.554406 10173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:02:25.555382 10173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:02:25.565191 10173 server_base.cc:1034] running on GCE node
W20260806 16:02:25.565141 10179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:25.565164 10178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:25.565146 10181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:25.565878 10173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:02:25.566481 10173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:02:25.567647 10173 hybrid_clock.cc:648] HybridClock initialized: now 1786032145567628 us; error 31 us; skew 500 ppm
Aug 06 16:02:25 dist-test-slave-tm1g krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032145, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:02:25.570209 10173 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 16:02:25.571094 10173 webserver.cc:460] Webserver started at http://127.25.31.131:41939/ using document root <none> and password file <none>
I20260806 16:02:25.571645 10173 fs_manager.cc:359] Metadata directory not provided
I20260806 16:02:25.571760 10173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:02:25.575762 10173 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:02:25.577855 10188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:25.579083 10173 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:02:25.579228 10173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0"
format_stamp: "Formatted at 2026-08-06 16:02:04 on dist-test-slave-tm1g"
server_key: "e8a5155af0a746bfbb1b244b99b30d4a"
server_key_iv: "eb8ea19ce677a8755edf2879bfd728a7"
server_key_version: "encryptionkey@0"
I20260806 16:02:25.579762 10173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:02:25.589828 10173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:02:25.592626 10173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:02:25.592837 10173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:02:25.593433 10173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:02:25.594343 10173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:02:25.594422 10173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:25.594489 10173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:02:25.594537 10173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:25.604067 10173 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:43571
I20260806 16:02:25.604085 10305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:43571 every 8 connection(s)
I20260806 16:02:25.605095 10173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:02:25.608348 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 10173
Aug 06 16:02:25 dist-test-slave-tm1g krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032145, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:02:25.617930 10310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.609002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56426 (local address 127.25.31.190:37419)
0806 16:02:25.609739 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:25.609742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:25.609751 (+ 9us) server_negotiation.cc:407] Connection header received
0806 16:02:25.609782 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:25.609784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:25.609815 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:25.609919 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:02:25.610781 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.611516 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.612410 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.612724 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.612846 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.614344 (+ 1498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:02:25.614349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:02:25.614352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:02:25.614383 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:02:25.616076 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:25.616565 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:25.616571 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:25.616574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:25.616666 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:25.616959 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:25.616963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:25.616965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:25.617201 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:02:25.617419 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:25.617573 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 16:02:25.617708 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":677,"thread_start_us":81,"threads_started":1}
I20260806 16:02:25.619724 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.606922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:45725 (local address 127.25.31.190:37419)
0806 16:02:25.607069 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:25.607074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:25.608129 (+ 1055us) server_negotiation.cc:407] Connection header received
0806 16:02:25.609103 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:25.609107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:25.609164 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:25.609260 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:02:25.610553 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.611172 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.612450 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.612707 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.612781 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.615631 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:02:25.615635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:02:25.615638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:02:25.615675 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:02:25.617268 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:25.617858 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:25.617861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:25.617862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:25.617938 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:25.618614 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:25.618617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:25.618618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:25.618784 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:02:25.618896 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:25.619467 (+ 571us) server_negotiation.cc:299] Negotiation successful
0806 16:02:25.619558 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":51}
I20260806 16:02:25.620604 10308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.607356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:45725)
0806 16:02:25.607984 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:25.608024 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:25.608853 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:25.609425 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:25.609432 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:25.609777 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:02:25.610406 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.610423 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.611278 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.611281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.612306 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.612315 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.612860 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.612864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.613050 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.613693 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:02:25.613708 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:02:25.615467 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:02:25.617388 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:25.617393 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:25.617407 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:25.617761 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:25.618250 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:25.618252 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:25.618253 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:25.618490 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:25.618897 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:02:25.618904 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:02:25.619213 (+ 309us) client_negotiation.cc:769] Sending connection context
0806 16:02:25.619412 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 16:02:25.619760 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":115,"threads_started":1}
I20260806 16:02:25.621644 10306 heartbeater.cc:347] Connected to a master server at 127.25.31.190:37419
I20260806 16:02:25.621853 10306 heartbeater.cc:464] Registering TS with master...
I20260806 16:02:25.622279 10306 heartbeater.cc:511] Master 127.25.31.190:37419 requested a full tablet report, sending...
I20260806 16:02:25.623291 9834 ts_manager.cc:194] Registered new tserver with Master: b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571)
I20260806 16:02:25.623852 9834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:45725
I20260806 16:02:25.631479 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:02:25.635622 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.632079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56432 (local address 127.25.31.190:37419)
0806 16:02:25.632228 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:25.632232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:25.632397 (+ 165us) server_negotiation.cc:407] Connection header received
0806 16:02:25.632611 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:25.632613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:25.632703 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:25.632809 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:02:25.633056 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.633565 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.634485 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.634701 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.634805 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.635110 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:02:25.635230 (+ 120us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 16:02:25.635482 (+ 252us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56432: token expired
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 16:02:25.635676 10030 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56432: token expired
W20260806 16:02:25.636032 9762 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.25.31.190:37419)
I20260806 16:02:25.642694 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.636391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56436 (local address 127.25.31.190:37419)
0806 16:02:25.636507 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:25.636511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:25.636522 (+ 11us) server_negotiation.cc:407] Connection header received
0806 16:02:25.636574 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:25.636576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:25.636607 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:25.636666 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:02:25.637248 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.637705 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.638526 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.638718 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.638795 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.639962 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:02:25.639965 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:02:25.639966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:02:25.639987 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:02:25.641332 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:25.641699 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:25.641701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:25.641702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:25.641772 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:25.642003 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:25.642005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:25.642006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:25.642151 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:02:25.642246 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:25.642508 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 16:02:25.642593 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 16:02:25.644830 9834 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:02:25.648041 9834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:02:25.655396 9960 tablet_service.cc:1467] Processing CreateTablet for tablet c040715d80dc47b4beed60997c10d1e7 (DEFAULT_TABLE table=test-table [id=7be00a05b4f6416ab04895526249ae70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:02:25.656080 9960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c040715d80dc47b4beed60997c10d1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:02:25.659047 9916 tablet_bootstrap.cc:492] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2: Bootstrap starting.
I20260806 16:02:25.659503 10034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.653975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:43571 (local address 127.0.0.1:50958)
0806 16:02:25.654102 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:25.654118 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:25.654191 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:25.655347 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:25.655350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:25.655387 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:02:25.655489 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.655495 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.656666 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.656671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.657884 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.657894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.659063 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.659067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.659177 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.659224 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 16:02:25.659282 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 16:02:25.659341 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"spinlock_wait_cycles":11008}
I20260806 16:02:25.659973 9916 tablet_bootstrap.cc:654] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2: Neither blocks nor log segments found. Creating new log.
I20260806 16:02:25.660151 10315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.654408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:45345 (local address 127.0.0.1:33726)
0806 16:02:25.655278 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:25.655293 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:25.655388 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:25.655904 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:25.655907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:25.655932 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:02:25.656022 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.656027 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.657605 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.657608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.658521 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.658531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.659694 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.659697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.659798 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.659822 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:02:25.659908 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 16:02:25.660009 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"spinlock_wait_cycles":3456,"thread_start_us":110,"threads_started":1}
I20260806 16:02:25.660274 10317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.654873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33726 (local address 127.25.31.130:45345)
0806 16:02:25.655207 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:25.655222 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:25.655487 (+ 265us) server_negotiation.cc:407] Connection header received
0806 16:02:25.655574 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:25.655578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:25.655759 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:25.655912 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:02:25.656181 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.657478 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.658638 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.659384 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.659488 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.659626 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:25.659967 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 16:02:25.660131 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":124,"threads_started":1}
I20260806 16:02:25.660506 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.654498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50958 (local address 127.25.31.131:43571)
0806 16:02:25.654831 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:25.654845 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:25.654870 (+ 25us) server_negotiation.cc:407] Connection header received
0806 16:02:25.654948 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:25.654956 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:25.655182 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:25.655357 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:02:25.655588 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.656551 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.658023 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.658907 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.660060 (+ 1153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.660199 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:25.660249 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 16:02:25.660354 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":99,"threads_started":1}
I20260806 16:02:25.661935 9916 tablet_bootstrap.cc:492] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2: No bootstrap required, opened a new log
I20260806 16:02:25.662030 9916 ts_tablet_manager.cc:1397] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 16:02:25.662518 9916 raft_consensus.cc:348] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:25.662644 9916 raft_consensus.cc:374] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:02:25.662693 9916 raft_consensus.cc:729] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72f7109b3a1b4a5e95fb1b831f1350f2, State: Initialized, Role: FOLLOWER
I20260806 16:02:25.662815 9916 consensus_queue.cc:260] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:25.663080 9916 ts_tablet_manager.cc:1428] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 16:02:25.663372 10101 tablet_service.cc:1467] Processing CreateTablet for tablet c040715d80dc47b4beed60997c10d1e7 (DEFAULT_TABLE table=test-table [id=7be00a05b4f6416ab04895526249ae70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:02:25.663383 10238 tablet_service.cc:1467] Processing CreateTablet for tablet c040715d80dc47b4beed60997c10d1e7 (DEFAULT_TABLE table=test-table [id=7be00a05b4f6416ab04895526249ae70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:02:25.664436 10101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c040715d80dc47b4beed60997c10d1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:02:25.664462 10238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c040715d80dc47b4beed60997c10d1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:02:25.671100 10319 tablet_bootstrap.cc:492] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0: Bootstrap starting.
I20260806 16:02:25.671545 10320 tablet_bootstrap.cc:492] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Bootstrap starting.
I20260806 16:02:25.673493 10320 tablet_bootstrap.cc:654] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Neither blocks nor log segments found. Creating new log.
I20260806 16:02:25.673489 10319 tablet_bootstrap.cc:654] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0: Neither blocks nor log segments found. Creating new log.
I20260806 16:02:25.674443 10320 log.cc:824] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Log is configured to *not* fsync() on all Append() calls
I20260806 16:02:25.674480 10319 log.cc:824] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0: Log is configured to *not* fsync() on all Append() calls
I20260806 16:02:25.676221 10320 tablet_bootstrap.cc:492] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: No bootstrap required, opened a new log
I20260806 16:02:25.676234 10319 tablet_bootstrap.cc:492] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0: No bootstrap required, opened a new log
I20260806 16:02:25.676432 10319 ts_tablet_manager.cc:1397] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260806 16:02:25.676440 10320 ts_tablet_manager.cc:1397] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 16:02:25.679208 10319 raft_consensus.cc:348] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:25.679222 10320 raft_consensus.cc:348] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:25.679507 10320 raft_consensus.cc:374] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:02:25.679508 10319 raft_consensus.cc:374] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:02:25.679572 10320 raft_consensus.cc:729] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82ecd0aa5a2e4a428ba6d37b4a0e1a75, State: Initialized, Role: FOLLOWER
I20260806 16:02:25.679572 10319 raft_consensus.cc:729] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4b44a0d86f64554b32e15d1b4fa8ba0, State: Initialized, Role: FOLLOWER
I20260806 16:02:25.680029 10319 consensus_queue.cc:260] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:25.680035 10320 consensus_queue.cc:260] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:25.680661 10170 heartbeater.cc:503] Master 127.25.31.190:37419 was elected leader, sending a full tablet report...
I20260806 16:02:25.680661 10306 heartbeater.cc:503] Master 127.25.31.190:37419 was elected leader, sending a full tablet report...
I20260806 16:02:25.681002 10319 ts_tablet_manager.cc:1428] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260806 16:02:25.681002 10320 ts_tablet_manager.cc:1428] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
W20260806 16:02:25.747745 10028 tablet.cc:2369] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:02:25.856148 10307 tablet.cc:2369] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:02:25.907892 10033 raft_consensus.cc:482] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:02:25.908124 10033 raft_consensus.cc:504] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:25.909255 10033 leader_election.cc:288] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571), 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345)
I20260806 16:02:25.914876 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.909623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:52347 (local address 127.25.31.131:43571)
0806 16:02:25.909761 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:25.909766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:25.909996 (+ 230us) server_negotiation.cc:407] Connection header received
0806 16:02:25.910191 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:25.910194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:25.910246 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:25.910415 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:02:25.910801 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.911616 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.913477 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.914080 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.914210 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.914261 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:25.914664 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 16:02:25.914745 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 16:02:25.914884 10317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.909629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:40931 (local address 127.25.31.130:45345)
0806 16:02:25.909761 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:25.909766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:25.910072 (+ 306us) server_negotiation.cc:407] Connection header received
0806 16:02:25.910191 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:25.910194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:25.910251 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:25.910389 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:02:25.910800 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.911597 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.913477 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.914080 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.914210 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.914261 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:25.914664 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 16:02:25.914745 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 16:02:25.914911 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.909513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:43571 (local address 127.25.31.129:52347)
0806 16:02:25.909895 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:25.909912 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:25.910052 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:25.910434 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:25.910437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:25.910483 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:02:25.910678 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.910685 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.911729 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.911733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.913325 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.913338 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.914213 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.914217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.914409 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.914519 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 16:02:25.914599 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 16:02:25.914723 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":284,"spinlock_wait_cycles":31104,"thread_start_us":93,"threads_started":1}
I20260806 16:02:25.914937 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.909513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:45345 (local address 127.25.31.129:40931)
0806 16:02:25.909991 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:25.910005 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:25.910078 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:25.910433 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:25.910437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:25.910484 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:02:25.910678 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.910685 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.911729 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.911733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.913325 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:25.913338 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.914213 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.914217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:25.914438 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.914513 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 16:02:25.914599 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 16:02:25.914723 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":353,"spinlock_wait_cycles":4992,"thread_start_us":72,"threads_started":1}
I20260806 16:02:25.915652 10242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c040715d80dc47b4beed60997c10d1e7" candidate_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" is_pre_election: true
I20260806 16:02:25.915889 10123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c040715d80dc47b4beed60997c10d1e7" candidate_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" is_pre_election: true
I20260806 16:02:25.916000 10242 raft_consensus.cc:2454] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72f7109b3a1b4a5e95fb1b831f1350f2 in term 0.
I20260806 16:02:25.916167 10123 raft_consensus.cc:2454] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72f7109b3a1b4a5e95fb1b831f1350f2 in term 0.
I20260806 16:02:25.916528 9912 leader_election.cc:302] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72f7109b3a1b4a5e95fb1b831f1350f2, b4b44a0d86f64554b32e15d1b4fa8ba0; no voters:
I20260806 16:02:25.916810 10033 raft_consensus.cc:2781] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:02:25.916904 10033 raft_consensus.cc:482] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:02:25.916949 10033 raft_consensus.cc:3037] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:02:25.917999 10033 raft_consensus.cc:504] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:25.918452 10033 leader_election.cc:288] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 1 election: Requested vote from peers b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571), 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345)
I20260806 16:02:25.918758 10242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c040715d80dc47b4beed60997c10d1e7" candidate_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0"
I20260806 16:02:25.918846 10123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c040715d80dc47b4beed60997c10d1e7" candidate_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75"
I20260806 16:02:25.918900 10242 raft_consensus.cc:3037] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:02:25.918992 10123 raft_consensus.cc:3037] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:02:25.919960 10242 raft_consensus.cc:2454] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72f7109b3a1b4a5e95fb1b831f1350f2 in term 1.
I20260806 16:02:25.920099 10123 raft_consensus.cc:2454] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72f7109b3a1b4a5e95fb1b831f1350f2 in term 1.
I20260806 16:02:25.920271 9912 leader_election.cc:302] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72f7109b3a1b4a5e95fb1b831f1350f2, b4b44a0d86f64554b32e15d1b4fa8ba0; no voters:
I20260806 16:02:25.920441 10033 raft_consensus.cc:2781] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:02:25.920603 10033 raft_consensus.cc:686] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 LEADER]: Becoming Leader. State: Replica: 72f7109b3a1b4a5e95fb1b831f1350f2, State: Running, Role: LEADER
I20260806 16:02:25.920853 10033 consensus_queue.cc:237] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:25.923804 9833 catalog_manager.cc:5495] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72f7109b3a1b4a5e95fb1b831f1350f2 (127.25.31.129). New cstate: current_term: 1 leader_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } health_report { overall_health: HEALTHY } } }
W20260806 16:02:25.929557 10171 tablet.cc:2369] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:02:25.953905 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.950377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43554 (local address 127.25.31.129:45833)
0806 16:02:25.950666 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:25.950673 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:25.950685 (+ 12us) server_negotiation.cc:407] Connection header received
0806 16:02:25.950754 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:25.950756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:25.950808 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:25.950908 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:02:25.951172 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.951688 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.952515 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:25.952727 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:25.952801 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:25.953096 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:02:25.953505 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:02:25.953587 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:25.953648 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 16:02:25.953694 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":114,"threads_started":1}
I20260806 16:02:25.957885 10123 raft_consensus.cc:1261] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 1 FOLLOWER]: Refusing update from remote peer 72f7109b3a1b4a5e95fb1b831f1350f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:02:25.957886 10242 raft_consensus.cc:1261] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 1 FOLLOWER]: Refusing update from remote peer 72f7109b3a1b4a5e95fb1b831f1350f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:02:25.958621 10033 consensus_queue.cc:1035] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:02:25.958835 10327 consensus_queue.cc:1035] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:02:25.965540 10331 mvcc.cc:205] Tried to move back new op lower bound from 7315587669836648448 to 7315587669694087168. Current Snapshot: MvccSnapshot[applied={T|T < 7315587669836648448}]
I20260806 16:02:25.966974 10333 mvcc.cc:205] Tried to move back new op lower bound from 7315587669836648448 to 7315587669694087168. Current Snapshot: MvccSnapshot[applied={T|T < 7315587669836648448}]
I20260806 16:02:25.967336 10332 mvcc.cc:205] Tried to move back new op lower bound from 7315587669836648448 to 7315587669694087168. Current Snapshot: MvccSnapshot[applied={T|T < 7315587669836648448}]
I20260806 16:02:25.972954 9833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:02:25.973093 9833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:02:25.975584 9833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Sending DeleteTablet for 3 replicas of tablet c040715d80dc47b4beed60997c10d1e7
I20260806 16:02:25.976478 9960 tablet_service.cc:1514] Processing DeleteTablet for tablet c040715d80dc47b4beed60997c10d1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:02:25 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:43552
I20260806 16:02:25.976554 10101 tablet_service.cc:1514] Processing DeleteTablet for tablet c040715d80dc47b4beed60997c10d1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:02:25 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:33726
I20260806 16:02:25.976984 10343 tablet_replica.cc:330] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: stopping tablet replica
I20260806 16:02:25.976984 10238 tablet_service.cc:1514] Processing DeleteTablet for tablet c040715d80dc47b4beed60997c10d1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:02:25 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:50958
I20260806 16:02:25.977226 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 9873
I20260806 16:02:25.977344 10343 raft_consensus.cc:2229] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:02:25.977528 10344 tablet_replica.cc:330] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0: stopping tablet replica
I20260806 16:02:25.978417 10344 raft_consensus.cc:2229] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:02:25.978782 10344 raft_consensus.cc:2258] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:02:25.978958 10343 raft_consensus.cc:2258] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:02:25.980338 10343 ts_tablet_manager.cc:1905] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:02:25.980360 10344 ts_tablet_manager.cc:1905] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:02:25.982661 10343 ts_tablet_manager.cc:1918] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:02:25.982774 10343 log.cc:1196] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/c040715d80dc47b4beed60997c10d1e7
I20260806 16:02:25.983153 10343 ts_tablet_manager.cc:1939] T c040715d80dc47b4beed60997c10d1e7 P 82ecd0aa5a2e4a428ba6d37b4a0e1a75: Deleting consensus metadata
I20260806 16:02:25.983256 10344 ts_tablet_manager.cc:1918] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:02:25.983362 10344 log.cc:1196] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/c040715d80dc47b4beed60997c10d1e7
I20260806 16:02:25.983740 10344 ts_tablet_manager.cc:1939] T c040715d80dc47b4beed60997c10d1e7 P b4b44a0d86f64554b32e15d1b4fa8ba0: Deleting consensus metadata
I20260806 16:02:25.984414 9821 catalog_manager.cc:4841] TS 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345): tablet c040715d80dc47b4beed60997c10d1e7 (table test-table [id=7be00a05b4f6416ab04895526249ae70]) successfully deleted
I20260806 16:02:25.984892 9820 catalog_manager.cc:4841] TS b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571): tablet c040715d80dc47b4beed60997c10d1e7 (table test-table [id=7be00a05b4f6416ab04895526249ae70]) successfully deleted
W20260806 16:02:25.985636 9821 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:45833 (error 108)
I20260806 16:02:25.986495 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 10031
I20260806 16:02:25.986755 10315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:25.986426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:45833 (local address 127.0.0.1:43558)
0806 16:02:25.986588 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:25.986662 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 16:02:25.987385 9821 catalog_manager.cc:4584] TS 72f7109b3a1b4a5e95fb1b831f1350f2 (127.25.31.129:45833): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c040715d80dc47b4beed60997c10d1e7: Network error: Client connection negotiation failed: client connection to 127.25.31.129:45833: connect: Connection refused (error 111)
I20260806 16:02:25.995208 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 10173
I20260806 16:02:26.001384 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 9801
I20260806 16:02:46.008826 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:37419
--webserver_interface=127.25.31.190
--webserver_port=35281
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:37419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:02:46.125537 10346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:02:46.125815 10346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:02:46.125895 10346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:02:46.129835 10346 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 16:02:46.129914 10346 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 16:02:46.129941 10346 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 16:02:46.129972 10346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:02:46.130023 10346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:02:46.130065 10346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:02:46.130118 10346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:02:46.130153 10346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:02:46.136184 10346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:37419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:37419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=35281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:02:46.137255 10346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:02:46.138293 10346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:02:46.147164 10354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:46.147320 10346 server_base.cc:1034] running on GCE node
W20260806 16:02:46.147176 10351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:46.147181 10352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:46.148236 10346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:02:46.149663 10346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:02:46.150941 10346 hybrid_clock.cc:648] HybridClock initialized: now 1786032166150907 us; error 46 us; skew 500 ppm
Aug 06 16:02:46 dist-test-slave-tm1g krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032166, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:02:46.157512 10346 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:02:46.158620 10346 webserver.cc:460] Webserver started at http://127.25.31.190:35281/ using document root <none> and password file <none>
I20260806 16:02:46.159304 10346 fs_manager.cc:359] Metadata directory not provided
I20260806 16:02:46.159435 10346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:02:46.163401 10346 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:02:46.165555 10361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:46.166783 10346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:02:46.166929 10346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "548f7867ad1e4b3fae33485cc0241dbc"
format_stamp: "Formatted at 2026-08-06 16:02:03 on dist-test-slave-tm1g"
server_key: "f46fee9c724fd4d131d4de27b0de32a7"
server_key_iv: "2cc28f646ba2d23b36d279a0e4876f11"
server_key_version: "encryptionkey@0"
I20260806 16:02:46.167476 10346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:02:46.193261 10346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:02:46.196595 10346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:02:46.196808 10346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:02:46.204169 10346 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:37419
I20260806 16:02:46.204473 10414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:37419 every 8 connection(s)
I20260806 16:02:46.205247 10346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:02:46.205557 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 10346
I20260806 16:02:46.206084 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:45833
--local_ip_for_outbound_sockets=127.25.31.129
--tserver_master_addrs=127.25.31.190:37419
--webserver_port=33319
--webserver_interface=127.25.31.129
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:02:46.209628 10415 sys_catalog.cc:263] Verifying existing consensus state
I20260806 16:02:46.211300 10415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Bootstrap starting.
I20260806 16:02:46.229272 10415 log.cc:824] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Log is configured to *not* fsync() on all Append() calls
I20260806 16:02:46.240275 10415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:02:46.240684 10415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: Bootstrap complete.
I20260806 16:02:46.244699 10415 raft_consensus.cc:348] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } }
I20260806 16:02:46.245069 10415 raft_consensus.cc:729] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 548f7867ad1e4b3fae33485cc0241dbc, State: Initialized, Role: FOLLOWER
I20260806 16:02:46.245589 10415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } }
I20260806 16:02:46.245709 10415 raft_consensus.cc:388] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:02:46.245793 10415 raft_consensus.cc:482] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:02:46.245910 10415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 2 FOLLOWER]: Advancing to term 3
I20260806 16:02:46.247781 10415 raft_consensus.cc:504] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } }
I20260806 16:02:46.248103 10415 leader_election.cc:302] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 548f7867ad1e4b3fae33485cc0241dbc; no voters:
I20260806 16:02:46.248399 10415 leader_election.cc:288] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 16:02:46.248559 10420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 3 FOLLOWER]: Leader election won for term 3
I20260806 16:02:46.248948 10420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [term 3 LEADER]: Becoming Leader. State: Replica: 548f7867ad1e4b3fae33485cc0241dbc, State: Running, Role: LEADER
I20260806 16:02:46.249352 10420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } }
I20260806 16:02:46.249583 10415 sys_catalog.cc:564] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:02:46.250896 10421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "548f7867ad1e4b3fae33485cc0241dbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } } }
I20260806 16:02:46.250939 10422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 548f7867ad1e4b3fae33485cc0241dbc. Latest consensus state: current_term: 3 leader_uuid: "548f7867ad1e4b3fae33485cc0241dbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "548f7867ad1e4b3fae33485cc0241dbc" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 37419 } } }
I20260806 16:02:46.251025 10421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [sys.catalog]: This master's current role is: LEADER
I20260806 16:02:46.251044 10422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc [sys.catalog]: This master's current role is: LEADER
I20260806 16:02:46.254660 10436 catalog_manager.cc:1259] Loaded cluster ID: 1cb2b5b9a9b644e599e9af3e7935c25c
I20260806 16:02:46.254719 10436 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: loading cluster ID for follower catalog manager: success
I20260806 16:02:46.257172 10436 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 548f7867ad1e4b3fae33485cc0241dbc: acquiring CA information for follower catalog manager: success
F20260806 16:02:46.258350 10436 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fab1f3489ad google::LogMessage::Fail() at ??:0
@ 0x7fab1f34b88a google::LogMessage::SendToLog() at ??:0
@ 0x7fab1f3484de google::LogMessage::Flush() at ??:0
@ 0x7fab1f349a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fab2200c18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fab29cb1989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7fab29c8e023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7fab29c8e54b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7fab29c87652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7fab29c869e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7fab29ccd541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fab2158b512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fab22bcf6db start_thread at ??:0
@ 0x7fab1de4661f clone at ??:0
W20260806 16:02:46.343314 10418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:02:46.343541 10418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:02:46.343582 10418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:02:46.347580 10418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:02:46.347636 10418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:02:46.347712 10418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:02:46.352802 10418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:45833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=33319
--tserver_master_addrs=127.25.31.190:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:02:46.353788 10418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:02:46.354841 10418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:02:46.365321 10457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:46.365374 10458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:46.365374 10461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:46.366032 10418 server_base.cc:1034] running on GCE node
I20260806 16:02:46.366539 10418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:02:46.367336 10418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:02:46.368553 10418 hybrid_clock.cc:648] HybridClock initialized: now 1786032166368519 us; error 47 us; skew 500 ppm
Aug 06 16:02:46 dist-test-slave-tm1g krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032166, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:02:46.371889 10418 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 16:02:46.372778 10418 webserver.cc:460] Webserver started at http://127.25.31.129:33319/ using document root <none> and password file <none>
I20260806 16:02:46.373322 10418 fs_manager.cc:359] Metadata directory not provided
I20260806 16:02:46.373382 10418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:02:46.377255 10418 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:02:46.379560 10468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:46.380801 10418 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:02:46.380934 10418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "72f7109b3a1b4a5e95fb1b831f1350f2"
format_stamp: "Formatted at 2026-08-06 16:02:04 on dist-test-slave-tm1g"
server_key: "96d56ebfb26229bd0b714a74bb36a79d"
server_key_iv: "9412760441f61205eee36a8e11a3b078"
server_key_version: "encryptionkey@0"
I20260806 16:02:46.381455 10418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:02:46.405020 10418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:02:46.408298 10418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:02:46.408500 10418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:02:46.409090 10418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:02:46.410193 10475 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 16:02:46.415656 10418 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 16:02:46.415736 10418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260806 16:02:46.415834 10418 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 16:02:46.417421 10418 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 16:02:46.417490 10418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 16:02:46.417860 10475 tablet_bootstrap.cc:492] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2: Bootstrap starting.
I20260806 16:02:46.428556 10418 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:45833
I20260806 16:02:46.428726 10585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:45833 every 8 connection(s)
I20260806 16:02:46.429529 10418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 16:02:46.433423 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 10418
I20260806 16:02:46.433851 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:45345
--local_ip_for_outbound_sockets=127.25.31.130
--tserver_master_addrs=127.25.31.190:37419
--webserver_port=41847
--webserver_interface=127.25.31.130
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:02:46.440533 10475 log.cc:824] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2: Log is configured to *not* fsync() on all Append() calls
I20260806 16:02:46.449661 10475 tablet_bootstrap.cc:492] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 16:02:46.450089 10475 tablet_bootstrap.cc:492] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2: Bootstrap complete.
I20260806 16:02:46.451607 10475 ts_tablet_manager.cc:1397] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2: Time spent bootstrapping tablet: real 0.034s user 0.024s sys 0.008s
I20260806 16:02:46.454842 10475 raft_consensus.cc:348] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:46.455394 10475 raft_consensus.cc:729] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72f7109b3a1b4a5e95fb1b831f1350f2, State: Initialized, Role: FOLLOWER
I20260806 16:02:46.455929 10475 consensus_queue.cc:260] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:46.457017 10475 ts_tablet_manager.cc:1428] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260806 16:02:46.474393 10588 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:02:46.431394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:47829)
0806 16:02:46.431859 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:46.431911 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:46.432706 (+ 795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:46.473463 (+ 40757us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":140,"threads_started":1}
W20260806 16:02:46.474980 10586 heartbeater.cc:644] Failed to heartbeat to 127.25.31.190:37419 (0 consecutive failures): Network error: Failed to ping master at 127.25.31.190:37419: Client connection negotiation failed: client connection to 127.25.31.190:37419: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 16:02:46.475749 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:46.475442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:58071)
0806 16:02:46.475602 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:46.475683 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 16:02:46.476429 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:46.476195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:50581)
0806 16:02:46.476305 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:46.476367 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 16:02:46.476630 10586 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:02:46.547216 10589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:02:46.547447 10589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:02:46.547487 10589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:02:46.551405 10589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:02:46.551455 10589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:02:46.551551 10589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:02:46.556542 10589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:45345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=41847
--tserver_master_addrs=127.25.31.190:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:02:46.557531 10589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:02:46.558524 10589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:02:46.568096 10596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:46.568084 10599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:46.568084 10597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:46.568135 10589 server_base.cc:1034] running on GCE node
I20260806 16:02:46.568717 10589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:02:46.569339 10589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:02:46.570525 10589 hybrid_clock.cc:648] HybridClock initialized: now 1786032166570502 us; error 43 us; skew 500 ppm
Aug 06 16:02:46 dist-test-slave-tm1g krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032166, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:02:46.573242 10589 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 16:02:46.574096 10589 webserver.cc:460] Webserver started at http://127.25.31.130:41847/ using document root <none> and password file <none>
I20260806 16:02:46.574714 10589 fs_manager.cc:359] Metadata directory not provided
I20260806 16:02:46.574810 10589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:02:46.578581 10589 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 16:02:46.580621 10606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:46.581996 10589 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:02:46.582132 10589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75"
format_stamp: "Formatted at 2026-08-06 16:02:04 on dist-test-slave-tm1g"
server_key: "aca0ae525da91ce01cfd608195826026"
server_key_iv: "8f77b5b23f7b22926c26eee09bdaf212"
server_key_version: "encryptionkey@0"
I20260806 16:02:46.582746 10589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:02:46.593677 10589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:02:46.596897 10589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:02:46.597113 10589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:02:46.597688 10589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:02:46.598584 10589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:02:46.598663 10589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:46.598757 10589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:02:46.598809 10589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:46.607821 10589 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:45345
I20260806 16:02:46.607856 10723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:45345 every 8 connection(s)
I20260806 16:02:46.608796 10589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:02:46.612211 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:46.610806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:44561)
0806 16:02:46.611443 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:46.611562 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":124,"threads_started":1}
I20260806 16:02:46.612453 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 10589
W20260806 16:02:46.612775 10724 heartbeater.cc:644] Failed to heartbeat to 127.25.31.190:37419 (0 consecutive failures): Network error: Failed to ping master at 127.25.31.190:37419: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
I20260806 16:02:46.612982 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:43571
--local_ip_for_outbound_sockets=127.25.31.131
--tserver_master_addrs=127.25.31.190:37419
--webserver_port=41939
--webserver_interface=127.25.31.131
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:02:46.613405 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:46.613183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:49369)
0806 16:02:46.613300 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:46.613353 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 16:02:46.614023 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:46.613817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:52281)
0806 16:02:46.613920 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:46.613963 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 16:02:46.614203 10724 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 16:02:46.722204 10727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:02:46.722517 10727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:02:46.722584 10727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:02:46.727147 10727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:02:46.727200 10727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:02:46.727290 10727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:02:46.732415 10727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:43571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=41939
--tserver_master_addrs=127.25.31.190:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:02:46.733397 10727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:02:46.734405 10727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:02:46.743702 10733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:46.743695 10732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:02:46.743700 10735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:02:46.743768 10727 server_base.cc:1034] running on GCE node
I20260806 16:02:46.744309 10727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:02:46.744957 10727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:02:46.746135 10727 hybrid_clock.cc:648] HybridClock initialized: now 1786032166746122 us; error 24 us; skew 500 ppm
Aug 06 16:02:46 dist-test-slave-tm1g krb5kdc[9233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032166, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:02:46.748931 10727 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 16:02:46.749711 10727 webserver.cc:460] Webserver started at http://127.25.31.131:41939/ using document root <none> and password file <none>
I20260806 16:02:46.750211 10727 fs_manager.cc:359] Metadata directory not provided
I20260806 16:02:46.750267 10727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:02:46.753927 10727 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:02:46.755973 10743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:46.757187 10727 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:02:46.757284 10727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0"
format_stamp: "Formatted at 2026-08-06 16:02:04 on dist-test-slave-tm1g"
server_key: "e8a5155af0a746bfbb1b244b99b30d4a"
server_key_iv: "eb8ea19ce677a8755edf2879bfd728a7"
server_key_version: "encryptionkey@0"
I20260806 16:02:46.757822 10727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:02:46.769493 10727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:02:46.772621 10727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:02:46.772811 10727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:02:46.773375 10727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:02:46.774163 10727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:02:46.774209 10727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:46.774274 10727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:02:46.774363 10727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:02:46.782913 10727 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:43571
I20260806 16:02:46.782943 10859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:43571 every 8 connection(s)
I20260806 16:02:46.783856 10727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:02:46.787626 10862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:46.786068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:38683)
0806 16:02:46.786791 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:46.786951 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":116,"threads_started":1}
W20260806 16:02:46.788141 10860 heartbeater.cc:644] Failed to heartbeat to 127.25.31.190:37419 (0 consecutive failures): Network error: Failed to ping master at 127.25.31.190:37419: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
I20260806 16:02:46.788736 10862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:46.788522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:41389)
0806 16:02:46.788632 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:46.788685 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 16:02:46.789331 10862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:46.789131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:42521)
0806 16:02:46.789234 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:46.789283 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 16:02:46.789575 10860 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 16:02:46.789615 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 10727
I20260806 16:02:47.477711 10864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:47.477240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:33191)
0806 16:02:47.477536 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:47.477597 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":82,"threads_started":1}
I20260806 16:02:47.615095 10865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:47.614710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:45777)
0806 16:02:47.614941 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:47.614994 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":68,"threads_started":1}
I20260806 16:02:47.790536 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:47.790092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:47677)
0806 16:02:47.790371 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:47.790423 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":70,"threads_started":1}
I20260806 16:02:47.894018 10867 raft_consensus.cc:482] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:02:47.894215 10867 raft_consensus.cc:504] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:47.895332 10867 leader_election.cc:288] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571), 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345)
Aug 06 16:02:47 dist-test-slave-tm1g krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032166, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.131@KRBTEST.COM
Aug 06 16:02:47 dist-test-slave-tm1g krb5kdc[9233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032166, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.130@KRBTEST.COM
I20260806 16:02:47.909088 10868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:47.895549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:43571 (local address 127.25.31.129:53587)
0806 16:02:47.896152 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:47.896164 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:47.896289 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:47.897617 (+ 1328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:47.897623 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:47.898209 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:02:47.898786 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:47.898800 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:47.900197 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:47.900200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:47.901207 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:47.901217 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:47.901880 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:47.901883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:47.902009 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:47.902657 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:02:47.902673 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:02:47.904336 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:02:47.906655 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:47.906661 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:47.906674 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:47.906981 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:47.907328 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:47.907331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:47.907333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:47.907482 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:47.908087 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:02:47.908090 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:02:47.908342 (+ 252us) client_negotiation.cc:769] Sending connection context
0806 16:02:47.908528 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 16:02:47.908856 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":387,"spinlock_wait_cycles":51712,"thread_start_us":97,"threads_started":1}
I20260806 16:02:47.909103 10869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:47.895763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:53587 (local address 127.25.31.131:43571)
0806 16:02:47.896039 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:47.896067 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:47.896270 (+ 203us) server_negotiation.cc:407] Connection header received
0806 16:02:47.896972 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:47.896977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:47.897263 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:47.897572 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:02:47.898953 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:47.900012 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:47.901322 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:47.901777 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:47.901898 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:47.904581 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:02:47.904585 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:02:47.904596 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:02:47.904622 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:02:47.906526 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:47.907095 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:47.907101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:47.907113 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:47.907219 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:47.907582 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:47.907585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:47.907587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:47.907971 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:02:47.908111 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:47.908640 (+ 529us) server_negotiation.cc:299] Negotiation successful
0806 16:02:47.908892 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":87,"threads_started":1}
I20260806 16:02:47.909121 10870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:47.895764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:44741 (local address 127.25.31.130:45345)
0806 16:02:47.896180 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0806 16:02:47.896200 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:02:47.896235 (+ 35us) server_negotiation.cc:407] Connection header received
0806 16:02:47.896923 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:02:47.896929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:02:47.897263 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:02:47.897573 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:02:47.898967 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:47.900020 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:47.902025 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:02:47.902531 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:02:47.902696 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:47.904914 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:02:47.904918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:02:47.904928 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:02:47.904954 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:02:47.906588 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:47.907094 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:47.907099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:47.907103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:47.907213 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:02:47.907582 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:02:47.907585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:02:47.907587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:02:47.907964 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:02:47.908109 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 16:02:47.908639 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 16:02:47.908924 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":78,"threads_started":1}
I20260806 16:02:47.909111 10864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:47.895548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:45345 (local address 127.25.31.129:44741)
0806 16:02:47.895905 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:47.895921 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:02:47.896030 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:02:47.897617 (+ 1587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:02:47.897623 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:02:47.898208 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:02:47.898786 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:47.898800 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:47.900196 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:47.900200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:47.901205 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:02:47.901215 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:02:47.902656 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:02:47.902660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:02:47.902741 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:02:47.903235 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:02:47.903246 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:02:47.904722 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:02:47.906701 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:47.906706 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:47.906708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:47.906971 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:47.907313 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:02:47.907317 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:02:47.907319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:02:47.907476 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:02:47.908084 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:02:47.908090 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:02:47.908350 (+ 260us) client_negotiation.cc:769] Sending connection context
0806 16:02:47.908528 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 16:02:47.908941 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":256,"spinlock_wait_cycles":16128}
W20260806 16:02:47.911897 10471 leader_election.cc:341] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571): Not found: Tablet not found: c040715d80dc47b4beed60997c10d1e7
W20260806 16:02:47.912070 10472 leader_election.cc:341] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345): Not found: Tablet not found: c040715d80dc47b4beed60997c10d1e7
I20260806 16:02:47.912168 10472 leader_election.cc:302] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72f7109b3a1b4a5e95fb1b831f1350f2; no voters: 82ecd0aa5a2e4a428ba6d37b4a0e1a75, b4b44a0d86f64554b32e15d1b4fa8ba0
I20260806 16:02:47.912384 10867 raft_consensus.cc:2726] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:02:48.478899 10871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:48.478493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:47083)
0806 16:02:48.478735 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:48.478788 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":75,"threads_started":1}
I20260806 16:02:48.616186 10872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:48.615809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:39133)
0806 16:02:48.616048 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:48.616103 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":75,"threads_started":1}
I20260806 16:02:48.791648 10873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:48.791214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:36485)
0806 16:02:48.791507 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:48.791562 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":111,"threads_started":1}
I20260806 16:02:49.480154 10874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:49.479704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:52955)
0806 16:02:49.479988 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:49.480054 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":84,"threads_started":1}
I20260806 16:02:49.617404 10875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:49.616956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:53753)
0806 16:02:49.617250 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:49.617304 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":135,"threads_started":1}
I20260806 16:02:49.792826 10876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:49.792442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:58829)
0806 16:02:49.792685 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:49.792740 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":73,"threads_started":1}
I20260806 16:02:49.817153 10877 raft_consensus.cc:482] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:02:49.817329 10877 raft_consensus.cc:504] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:49.817801 10877 leader_election.cc:288] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571), 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345)
W20260806 16:02:49.818663 10472 leader_election.cc:341] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345): Not found: Tablet not found: c040715d80dc47b4beed60997c10d1e7
W20260806 16:02:49.818781 10471 leader_election.cc:341] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571): Not found: Tablet not found: c040715d80dc47b4beed60997c10d1e7
I20260806 16:02:49.818895 10471 leader_election.cc:302] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72f7109b3a1b4a5e95fb1b831f1350f2; no voters: 82ecd0aa5a2e4a428ba6d37b4a0e1a75, b4b44a0d86f64554b32e15d1b4fa8ba0
I20260806 16:02:49.819056 10877 raft_consensus.cc:2726] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:02:50.481354 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:50.480903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:40793)
0806 16:02:50.481193 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:50.481253 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":90,"threads_started":1}
I20260806 16:02:50.618669 10879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:50.618233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:38993)
0806 16:02:50.618523 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:50.618578 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":131,"threads_started":1}
I20260806 16:02:50.793926 10880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:50.793514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:45175)
0806 16:02:50.793776 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:50.793829 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":89,"threads_started":1}
I20260806 16:02:51.482798 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:51.482261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:48431)
0806 16:02:51.482637 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:51.482702 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":88,"threads_started":1}
I20260806 16:02:51.619905 10882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:51.619427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:54411)
0806 16:02:51.619735 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:51.619811 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":78,"threads_started":1}
I20260806 16:02:51.795126 10883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:51.794725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:33849)
0806 16:02:51.794979 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:51.795034 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":72,"threads_started":1}
I20260806 16:02:51.907980 10884 raft_consensus.cc:482] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:02:51.908151 10884 raft_consensus.cc:504] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:51.908695 10884 leader_election.cc:288] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571), 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345)
W20260806 16:02:51.909435 10471 leader_election.cc:341] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571): Not found: Tablet not found: c040715d80dc47b4beed60997c10d1e7
W20260806 16:02:51.909662 10472 leader_election.cc:341] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345): Not found: Tablet not found: c040715d80dc47b4beed60997c10d1e7
I20260806 16:02:51.909759 10472 leader_election.cc:302] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72f7109b3a1b4a5e95fb1b831f1350f2; no voters: 82ecd0aa5a2e4a428ba6d37b4a0e1a75, b4b44a0d86f64554b32e15d1b4fa8ba0
I20260806 16:02:51.909925 10884 raft_consensus.cc:2726] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:02:52.483937 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:52.483519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:46327)
0806 16:02:52.483773 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:52.483835 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":90,"threads_started":1}
I20260806 16:02:52.621150 10887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:52.620699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:34193)
0806 16:02:52.620995 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:52.621054 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":96,"threads_started":1}
I20260806 16:02:52.796209 10888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:52.795788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:43229)
0806 16:02:52.796067 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:52.796120 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":94,"threads_started":1}
I20260806 16:02:53.485180 10889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:53.484745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:60597)
0806 16:02:53.485026 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:53.485081 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":85,"threads_started":1}
I20260806 16:02:53.622519 10890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:53.621933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:45323)
0806 16:02:53.622273 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:53.622419 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":91,"threads_started":1}
I20260806 16:02:53.797459 10891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:53.796968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:60855)
0806 16:02:53.797291 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:53.797348 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":86,"threads_started":1}
I20260806 16:02:54.172962 10892 raft_consensus.cc:482] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:02:54.173098 10892 raft_consensus.cc:504] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:54.173614 10892 leader_election.cc:288] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571), 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345)
W20260806 16:02:54.174430 10471 leader_election.cc:341] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571): Not found: Tablet not found: c040715d80dc47b4beed60997c10d1e7
W20260806 16:02:54.174593 10472 leader_election.cc:341] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345): Not found: Tablet not found: c040715d80dc47b4beed60997c10d1e7
I20260806 16:02:54.174700 10472 leader_election.cc:302] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72f7109b3a1b4a5e95fb1b831f1350f2; no voters: 82ecd0aa5a2e4a428ba6d37b4a0e1a75, b4b44a0d86f64554b32e15d1b4fa8ba0
I20260806 16:02:54.174880 10892 raft_consensus.cc:2726] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:02:54.486454 10893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:54.485937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:44045)
0806 16:02:54.486234 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:54.486287 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":107,"threads_started":1}
I20260806 16:02:54.623673 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:54.623268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:35759)
0806 16:02:54.623525 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:54.623580 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
I20260806 16:02:54.798751 10895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:54.798242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:35923)
0806 16:02:54.798542 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:54.798616 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":144,"threads_started":1}
I20260806 16:02:55.487618 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:55.487175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:49049)
0806 16:02:55.487456 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:55.487520 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":92,"threads_started":1}
I20260806 16:02:55.624904 10897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:55.624468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:58863)
0806 16:02:55.624743 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:55.624801 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":87,"threads_started":1}
I20260806 16:02:55.799896 10898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:55.799452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:52463)
0806 16:02:55.799752 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:55.799806 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":95,"threads_started":1}
I20260806 16:02:55.951700 10899 raft_consensus.cc:482] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:02:55.951881 10899 raft_consensus.cc:504] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:55.952373 10899 leader_election.cc:288] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571), 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345)
W20260806 16:02:55.953122 10471 leader_election.cc:341] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571): Not found: Tablet not found: c040715d80dc47b4beed60997c10d1e7
W20260806 16:02:55.953397 10472 leader_election.cc:341] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345): Not found: Tablet not found: c040715d80dc47b4beed60997c10d1e7
I20260806 16:02:55.953469 10472 leader_election.cc:302] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72f7109b3a1b4a5e95fb1b831f1350f2; no voters: 82ecd0aa5a2e4a428ba6d37b4a0e1a75, b4b44a0d86f64554b32e15d1b4fa8ba0
I20260806 16:02:55.953604 10899 raft_consensus.cc:2726] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:02:56.488881 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:56.488463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:60955)
0806 16:02:56.488734 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:56.488787 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":79,"threads_started":1}
I20260806 16:02:56.626052 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:56.625656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:55309)
0806 16:02:56.625905 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:56.625958 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":86,"threads_started":1}
I20260806 16:02:56.801040 10902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:56.800652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:47459)
0806 16:02:56.800892 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:56.800943 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":83,"threads_started":1}
I20260806 16:02:57.490047 10903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:57.489648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:41443)
0806 16:02:57.489895 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:57.489950 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":76,"threads_started":1}
I20260806 16:02:57.627200 10904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:57.626780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:44669)
0806 16:02:57.627048 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:57.627105 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":80,"threads_started":1}
I20260806 16:02:57.662003 10905 raft_consensus.cc:482] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:02:57.662142 10905 raft_consensus.cc:504] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b44a0d86f64554b32e15d1b4fa8ba0" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 43571 } } peers { permanent_uuid: "82ecd0aa5a2e4a428ba6d37b4a0e1a75" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 45345 } } peers { permanent_uuid: "72f7109b3a1b4a5e95fb1b831f1350f2" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 45833 } }
I20260806 16:02:57.662598 10905 leader_election.cc:288] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571), 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345)
W20260806 16:02:57.663390 10472 leader_election.cc:341] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 82ecd0aa5a2e4a428ba6d37b4a0e1a75 (127.25.31.130:45345): Not found: Tablet not found: c040715d80dc47b4beed60997c10d1e7
W20260806 16:02:57.663555 10471 leader_election.cc:341] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4b44a0d86f64554b32e15d1b4fa8ba0 (127.25.31.131:43571): Not found: Tablet not found: c040715d80dc47b4beed60997c10d1e7
I20260806 16:02:57.663647 10471 leader_election.cc:302] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 72f7109b3a1b4a5e95fb1b831f1350f2; no voters: 82ecd0aa5a2e4a428ba6d37b4a0e1a75, b4b44a0d86f64554b32e15d1b4fa8ba0
I20260806 16:02:57.663815 10905 raft_consensus.cc:2726] T c040715d80dc47b4beed60997c10d1e7 P 72f7109b3a1b4a5e95fb1b831f1350f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 16:02:57.802203 10906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:57.801793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:33551)
0806 16:02:57.802044 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:57.802108 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":75,"threads_started":1}
I20260806 16:02:58.491267 10907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:58.490829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:43725)
0806 16:02:58.491107 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:58.491164 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":92,"threads_started":1}
I20260806 16:02:58.628374 10908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:58.627965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:40809)
0806 16:02:58.628217 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:58.628271 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":76,"threads_started":1}
I20260806 16:02:58.803328 10909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:58.802897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:58561)
0806 16:02:58.803179 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:58.803237 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":98,"threads_started":1}
I20260806 16:02:59.492398 10910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:59.491986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:34935)
0806 16:02:59.492224 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:59.492279 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":75,"threads_started":1}
I20260806 16:02:59.629554 10911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:59.629149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:60153)
0806 16:02:59.629393 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:59.629449 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
I20260806 16:02:59.804471 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:02:59.804044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:52093)
0806 16:02:59.804311 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 16:02:59.804377 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":77,"threads_started":1}
I20260806 16:03:00.493587 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:00.493167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:42107)
0806 16:03:00.493429 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:00.493491 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":77,"threads_started":1}
I20260806 16:03:00.630836 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:00.630414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:56243)
0806 16:03:00.630687 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:00.630742 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":91,"threads_started":1}
I20260806 16:03:00.805564 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:00.805177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:56941)
0806 16:03:00.805419 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:00.805473 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
I20260806 16:03:01.494977 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:01.494588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.129:59567)
0806 16:03:01.494828 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:01.494882 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":74,"threads_started":1}
I20260806 16:03:01.632048 10917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:01.631662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.130:37407)
0806 16:03:01.631900 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:01.631954 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":72,"threads_started":1}
I20260806 16:03:01.798018 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 10418
I20260806 16:03:01.804903 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 10589
I20260806 16:03:01.807163 10918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:01.806766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:37419 (local address 127.25.31.131:41761)
0806 16:03:01.807019 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:01.807072 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.190:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":78,"threads_started":1}
I20260806 16:03:01.811126 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 10727
2026-08-06T16:03:01Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59215 ms)
[----------] 30 tests from SecurityITest (242219 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:01 dist-test-slave-tm1g krb5kdc[10921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:01 dist-test-slave-tm1g krb5kdc[10921](info): set up 2 sockets
Aug 06 16:03:01 dist-test-slave-tm1g krb5kdc[10921](info): commencing operation
krb5kdc: starting...
W20260806 16:03:02.875136 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:02 dist-test-slave-tm1g krb5kdc[10921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:03:02.993486 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:35505
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:37249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:35505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:03.100917 10934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:03.101207 10934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:03.101280 10934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:03.105212 10934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:03.105315 10934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:03.105355 10934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:03.105434 10934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:03.105487 10934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:03.110570 10934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:37249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:35505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:35505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:03.111589 10934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:03.112586 10934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:03.121173 10939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:03.121237 10942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:03.121297 10934 server_base.cc:1034] running on GCE node
W20260806 16:03:03.121237 10940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:03.122010 10934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:03.123065 10934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:03.124258 10934 hybrid_clock.cc:648] HybridClock initialized: now 1786032183124230 us; error 38 us; skew 500 ppm
Aug 06 16:03:03 dist-test-slave-tm1g krb5kdc[10921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032183, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:03.126991 10934 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:03:03.127873 10934 webserver.cc:460] Webserver started at http://127.25.31.190:39069/ using document root <none> and password file <none>
I20260806 16:03:03.128414 10934 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:03.128504 10934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:03.128741 10934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:03.130719 10934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "44011db6aa87487582a3fcbb5d23cc8b"
format_stamp: "Formatted at 2026-08-06 16:03:03 on dist-test-slave-tm1g"
server_key: "25da00689b1610bd798fc0d738f93f3f"
server_key_iv: "a8a48d724488bee5445f11699f520464"
server_key_version: "encryptionkey@0"
I20260806 16:03:03.131217 10934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "44011db6aa87487582a3fcbb5d23cc8b"
format_stamp: "Formatted at 2026-08-06 16:03:03 on dist-test-slave-tm1g"
server_key: "25da00689b1610bd798fc0d738f93f3f"
server_key_iv: "a8a48d724488bee5445f11699f520464"
server_key_version: "encryptionkey@0"
I20260806 16:03:03.134682 10934 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:03:03.137074 10949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:03.138227 10934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:03:03.138393 10934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "44011db6aa87487582a3fcbb5d23cc8b"
format_stamp: "Formatted at 2026-08-06 16:03:03 on dist-test-slave-tm1g"
server_key: "25da00689b1610bd798fc0d738f93f3f"
server_key_iv: "a8a48d724488bee5445f11699f520464"
server_key_version: "encryptionkey@0"
I20260806 16:03:03.138484 10934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:03.154974 10934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:03.157549 10934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:03.157692 10934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:03.164793 10934 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:35505
I20260806 16:03:03.164816 11002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:35505 every 8 connection(s)
I20260806 16:03:03.165721 10934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:03.168421 11003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:03.169337 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 10934
I20260806 16:03:03.169492 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:03.169793 25726 external_mini_cluster.cc:1428] Setting key 0ff02a42b13c3a9753a5eafd12d31515
I20260806 16:03:03.174571 11003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b: Bootstrap starting.
I20260806 16:03:03.177138 11003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:03.178073 11003 log.cc:824] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:03:03 dist-test-slave-tm1g krb5kdc[10921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:03:03.179903 11003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b: No bootstrap required, opened a new log
I20260806 16:03:03.183349 11003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44011db6aa87487582a3fcbb5d23cc8b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35505 } }
I20260806 16:03:03.183606 11003 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:03.183693 11003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44011db6aa87487582a3fcbb5d23cc8b, State: Initialized, Role: FOLLOWER
I20260806 16:03:03.184018 11006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:03.171263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33802 (local address 127.25.31.190:35505)
0806 16:03:03.171662 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:03.171678 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:03.171711 (+ 33us) server_negotiation.cc:407] Connection header received
0806 16:03:03.172308 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:03.172323 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:03.172635 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:03.172892 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:03.173785 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.174837 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.175987 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.176435 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.176545 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:03.179317 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:03.179322 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:03.179334 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:03.179360 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:03.181090 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:03.181552 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:03.181558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:03.181562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:03.181669 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:03.181956 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:03.181961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:03.181962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:03.182394 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:03.182602 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:03.182850 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 16:03:03.183134 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":137,"threads_started":1}
I20260806 16:03:03.184207 11003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44011db6aa87487582a3fcbb5d23cc8b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35505 } }
I20260806 16:03:03.184387 11003 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:03.184461 11003 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:03.184563 11003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:03.185608 11003 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44011db6aa87487582a3fcbb5d23cc8b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35505 } }
I20260806 16:03:03.185950 11003 leader_election.cc:302] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44011db6aa87487582a3fcbb5d23cc8b; no voters:
I20260806 16:03:03.186290 11003 leader_election.cc:288] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:03.186455 11008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:03.186684 11008 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [term 1 LEADER]: Becoming Leader. State: Replica: 44011db6aa87487582a3fcbb5d23cc8b, State: Running, Role: LEADER
I20260806 16:03:03.187004 11008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44011db6aa87487582a3fcbb5d23cc8b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35505 } }
I20260806 16:03:03.187561 11003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:03.188627 11010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44011db6aa87487582a3fcbb5d23cc8b. Latest consensus state: current_term: 1 leader_uuid: "44011db6aa87487582a3fcbb5d23cc8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44011db6aa87487582a3fcbb5d23cc8b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35505 } } }
I20260806 16:03:03.188755 11010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:03.189155 11017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:03.190186 11009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44011db6aa87487582a3fcbb5d23cc8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44011db6aa87487582a3fcbb5d23cc8b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35505 } } }
I20260806 16:03:03.190301 11009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:03.192157 11017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:03.197404 11017 catalog_manager.cc:1347] Generated new cluster ID: 770831171512427798d1987646e5261b
I20260806 16:03:03.197469 11017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:03.226241 11017 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:03.227258 11017 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:03.238219 11017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b: Generated new TSK 0
I20260806 16:03:03.238938 11017 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 16:03:03.274977 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35505
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:37249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:03.385013 11029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:03.385255 11029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:03.385321 11029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:03.389464 11029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:03.389519 11029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:03.389611 11029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:03:03.394774 11029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:37249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:03.395749 11029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:03.396742 11029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:03.406181 11035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:03.406145 11034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:03.406162 11037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:03.406404 11029 server_base.cc:1034] running on GCE node
I20260806 16:03:03.406852 11029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:03.407446 11029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:03.408617 11029 hybrid_clock.cc:648] HybridClock initialized: now 1786032183408589 us; error 40 us; skew 500 ppm
Aug 06 16:03:03 dist-test-slave-tm1g krb5kdc[10921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032183, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:03.411599 11029 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 16:03:03.412371 11029 webserver.cc:460] Webserver started at http://127.25.31.129:46347/ using document root <none> and password file <none>
I20260806 16:03:03.412869 11029 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:03.412925 11029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:03.413084 11029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:03.415064 11029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "3a0e4e7156e34c219bc7f4101edb02c0"
format_stamp: "Formatted at 2026-08-06 16:03:03 on dist-test-slave-tm1g"
server_key: "68f83d09d51c5e0bf28e927992f8c6a5"
server_key_iv: "5f5e9b54630a24f5888683785ccd72c5"
server_key_version: "encryptionkey@0"
I20260806 16:03:03.415573 11029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "3a0e4e7156e34c219bc7f4101edb02c0"
format_stamp: "Formatted at 2026-08-06 16:03:03 on dist-test-slave-tm1g"
server_key: "68f83d09d51c5e0bf28e927992f8c6a5"
server_key_iv: "5f5e9b54630a24f5888683785ccd72c5"
server_key_version: "encryptionkey@0"
I20260806 16:03:03.419077 11029 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:03:03.421430 11044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:03.422633 11029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:03:03.422756 11029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "3a0e4e7156e34c219bc7f4101edb02c0"
format_stamp: "Formatted at 2026-08-06 16:03:03 on dist-test-slave-tm1g"
server_key: "68f83d09d51c5e0bf28e927992f8c6a5"
server_key_iv: "5f5e9b54630a24f5888683785ccd72c5"
server_key_version: "encryptionkey@0"
I20260806 16:03:03.422865 11029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:03.444151 11029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:03.447256 11029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:03.447458 11029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:03.448021 11029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:03.448808 11029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:03.448854 11029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:03.448918 11029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:03.448951 11029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:03.458024 11029 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:35145
I20260806 16:03:03.458052 11158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:35145 every 8 connection(s)
I20260806 16:03:03.458976 11029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 16:03:03.460986 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 11029
I20260806 16:03:03.461124 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 16:03:03.461474 25726 external_mini_cluster.cc:1428] Setting key 42d21723ff367421d8a4b853b8d2ec8f
Aug 06 16:03:03 dist-test-slave-tm1g krb5kdc[10921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032183, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
I20260806 16:03:03.475914 11006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:03.461178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:60915 (local address 127.25.31.190:35505)
0806 16:03:03.461689 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:03.461696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:03.463330 (+ 1634us) server_negotiation.cc:407] Connection header received
0806 16:03:03.465340 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:03.465345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:03.465415 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:03.465504 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:03.466697 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.467260 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.469113 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.469343 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.469432 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:03.472278 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:03.472284 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:03.472290 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:03.472363 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:03.473794 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:03.474292 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:03.474295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:03.474296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:03.474442 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:03.474834 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:03.474844 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:03.474845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:03.475023 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:03.475126 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:03.475687 (+ 561us) server_negotiation.cc:299] Negotiation successful
0806 16:03:03.475792 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 16:03:03.477155 11161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:03.461451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:35505 (local address 127.25.31.129:60915)
0806 16:03:03.463158 (+ 1707us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:03.463217 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:03.465080 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:03.465640 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:03.465646 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:03.465995 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:03.466560 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:03.466576 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.467404 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.467410 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:03.468985 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:03.468999 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.469509 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.469515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:03.469762 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:03.470421 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:03.470445 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:03.472105 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:03.473907 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:03.473913 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:03.473924 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:03.474166 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:03.474550 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:03.474553 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:03.474555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:03.474738 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:03.475128 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:03.475139 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:03.475413 (+ 274us) client_negotiation.cc:769] Sending connection context
0806 16:03:03.475615 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 16:03:03.476047 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1448,"thread_start_us":136,"threads_started":1}
I20260806 16:03:03.478554 11159 heartbeater.cc:347] Connected to a master server at 127.25.31.190:35505
I20260806 16:03:03.478801 11159 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:03.479281 11159 heartbeater.cc:511] Master 127.25.31.190:35505 requested a full tablet report, sending...
I20260806 16:03:03.480985 10967 ts_manager.cc:194] Registered new tserver with Master: 3a0e4e7156e34c219bc7f4101edb02c0 (127.25.31.129:35145)
I20260806 16:03:03.482677 10967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:60915
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 16:03:03.492426 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35505
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:37249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:03.600512 11164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:03.600745 11164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:03.600786 11164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:03.604610 11164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:03.604655 11164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:03.604708 11164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:03:03.609795 11164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:37249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:03.610697 11164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:03.611611 11164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:03:03.621063 11164 server_base.cc:1034] running on GCE node
W20260806 16:03:03.620990 11172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:03.620977 11169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:03.620965 11170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:03.621640 11164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:03.622184 11164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:03.623334 11164 hybrid_clock.cc:648] HybridClock initialized: now 1786032183623323 us; error 29 us; skew 500 ppm
Aug 06 16:03:03 dist-test-slave-tm1g krb5kdc[10921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032183, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:03.625983 11164 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 16:03:03.626987 11164 webserver.cc:460] Webserver started at http://127.25.31.130:40261/ using document root <none> and password file <none>
I20260806 16:03:03.627566 11164 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:03.627642 11164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:03.627863 11164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:03.629810 11164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "9ae6440f099445f8b064d75e6e7d4b82"
format_stamp: "Formatted at 2026-08-06 16:03:03 on dist-test-slave-tm1g"
server_key: "16223d3549b2e76f600ea0bd5f7e5d52"
server_key_iv: "1daa69126a0308914470799f7896ab2e"
server_key_version: "encryptionkey@0"
I20260806 16:03:03.630440 11164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "9ae6440f099445f8b064d75e6e7d4b82"
format_stamp: "Formatted at 2026-08-06 16:03:03 on dist-test-slave-tm1g"
server_key: "16223d3549b2e76f600ea0bd5f7e5d52"
server_key_iv: "1daa69126a0308914470799f7896ab2e"
server_key_version: "encryptionkey@0"
I20260806 16:03:03.633980 11164 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:03:03.636300 11179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:03.637548 11164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 16:03:03.637670 11164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "9ae6440f099445f8b064d75e6e7d4b82"
format_stamp: "Formatted at 2026-08-06 16:03:03 on dist-test-slave-tm1g"
server_key: "16223d3549b2e76f600ea0bd5f7e5d52"
server_key_iv: "1daa69126a0308914470799f7896ab2e"
server_key_version: "encryptionkey@0"
I20260806 16:03:03.637771 11164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:03.651568 11164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:03.654409 11164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:03.654556 11164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:03.655049 11164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:03.655927 11164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:03.655979 11164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:03.656018 11164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:03.656032 11164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:03.664842 11164 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:40263
I20260806 16:03:03.664875 11294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:40263 every 8 connection(s)
I20260806 16:03:03.665706 11164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:03:03.668296 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 11164
I20260806 16:03:03.668452 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 16:03:03.668800 25726 external_mini_cluster.cc:1428] Setting key 3c08171f6398cd454a248a9775547778
Aug 06 16:03:03 dist-test-slave-tm1g krb5kdc[10921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032183, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
I20260806 16:03:03.681491 11006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:03.667473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:43419 (local address 127.25.31.190:35505)
0806 16:03:03.667622 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:03.667626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:03.668913 (+ 1287us) server_negotiation.cc:407] Connection header received
0806 16:03:03.670552 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:03.670555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:03.670598 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:03.670667 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:03.672174 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.673087 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.674752 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.674967 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.675051 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:03.677874 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:03.677878 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:03.677880 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:03.677901 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:03.679482 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:03.679974 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:03.679980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:03.679985 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:03.680067 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:03.680452 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:03.680455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:03.680456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:03.680639 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:03.680723 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:03.681167 (+ 444us) server_negotiation.cc:299] Negotiation successful
0806 16:03:03.681303 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
I20260806 16:03:03.682345 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:03.667863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:35505 (local address 127.25.31.130:43419)
0806 16:03:03.668719 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:03.668777 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:03.670268 (+ 1491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:03.670845 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:03.670855 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:03.671358 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:03.672020 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:03.672042 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.673238 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.673241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:03.674629 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:03.674645 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.675123 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.675130 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:03.675380 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:03.676004 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:03.676030 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:03.677707 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:03.679595 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:03.679601 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:03.679614 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:03.679870 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:03.680160 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:03.680164 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:03.680166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:03.680344 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:03.680736 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:03.680741 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:03.680960 (+ 219us) client_negotiation.cc:769] Sending connection context
0806 16:03:03.681118 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 16:03:03.681425 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":118,"threads_started":1}
I20260806 16:03:03.683602 11295 heartbeater.cc:347] Connected to a master server at 127.25.31.190:35505
I20260806 16:03:03.683856 11295 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:03.684336 11295 heartbeater.cc:511] Master 127.25.31.190:35505 requested a full tablet report, sending...
I20260806 16:03:03.685428 10967 ts_manager.cc:194] Registered new tserver with Master: 9ae6440f099445f8b064d75e6e7d4b82 (127.25.31.130:40263)
I20260806 16:03:03.686250 10967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:43419
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 16:03:03.698725 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35505
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:37249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:03.811966 11300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:03.812201 11300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:03.812242 11300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:03.816776 11300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:03.816828 11300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:03.816884 11300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:03:03.822149 11300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:37249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:03.823125 11300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:03.824108 11300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:03.833382 11306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:03.833425 11308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:03.833400 11305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:03.833715 11300 server_base.cc:1034] running on GCE node
I20260806 16:03:03.834213 11300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:03.834841 11300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:03.836027 11300 hybrid_clock.cc:648] HybridClock initialized: now 1786032183836017 us; error 29 us; skew 500 ppm
Aug 06 16:03:03 dist-test-slave-tm1g krb5kdc[10921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032183, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:03.838657 11300 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 16:03:03.839437 11300 webserver.cc:460] Webserver started at http://127.25.31.131:33647/ using document root <none> and password file <none>
I20260806 16:03:03.839926 11300 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:03.839988 11300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:03.840143 11300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:03.842338 11300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c"
format_stamp: "Formatted at 2026-08-06 16:03:03 on dist-test-slave-tm1g"
server_key: "53b7948c37b3619c1e9d0f90f88bea58"
server_key_iv: "365840a7ab6118daf5cc468d3955ec8e"
server_key_version: "encryptionkey@0"
I20260806 16:03:03.842914 11300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c"
format_stamp: "Formatted at 2026-08-06 16:03:03 on dist-test-slave-tm1g"
server_key: "53b7948c37b3619c1e9d0f90f88bea58"
server_key_iv: "365840a7ab6118daf5cc468d3955ec8e"
server_key_version: "encryptionkey@0"
I20260806 16:03:03.846414 11300 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:03:03.848608 11315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:03.849804 11300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:03:03.849925 11300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c"
format_stamp: "Formatted at 2026-08-06 16:03:03 on dist-test-slave-tm1g"
server_key: "53b7948c37b3619c1e9d0f90f88bea58"
server_key_iv: "365840a7ab6118daf5cc468d3955ec8e"
server_key_version: "encryptionkey@0"
I20260806 16:03:03.850034 11300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:03.867187 11300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:03.869712 11300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:03.869894 11300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:03.870498 11300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:03.871270 11300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:03.871317 11300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:03.871383 11300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:03.871428 11300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:03.880462 11300 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:35445
I20260806 16:03:03.880483 11432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:35445 every 8 connection(s)
I20260806 16:03:03.881295 11300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:03:03.884920 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 11300
I20260806 16:03:03.885049 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 16:03:03.885399 25726 external_mini_cluster.cc:1428] Setting key 799dbea61d994bb634b725bad2a1c072
Aug 06 16:03:03 dist-test-slave-tm1g krb5kdc[10921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032183, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:03:03.894845 11006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:03.883066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:41723 (local address 127.25.31.190:35505)
0806 16:03:03.883200 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:03.883203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:03.883942 (+ 739us) server_negotiation.cc:407] Connection header received
0806 16:03:03.885618 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:03.885621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:03.885675 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:03.885784 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:03.887253 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.887955 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.889180 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.889391 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.889470 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:03.891855 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:03.891858 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:03.891860 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:03.891884 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:03.893091 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:03.893547 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:03.893550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:03.893551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:03.893624 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:03.893910 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:03.893912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:03.893913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:03.894073 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:03.894147 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:03.894634 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 16:03:03.894728 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 16:03:03.895639 11435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:03.883353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:35505 (local address 127.25.31.131:41723)
0806 16:03:03.883792 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:03.883832 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:03.885319 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:03.885968 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:03.885977 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:03.886475 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:03.887114 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:03.887139 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.888122 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.888129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:03.889047 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:03.889057 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.889514 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.889517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:03.889652 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:03.890169 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:03.890186 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:03.891711 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:03.893203 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:03.893208 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:03.893217 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:03.893461 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:03.893703 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:03.893705 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:03.893706 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:03.893831 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:03.894159 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:03.894163 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:03.894434 (+ 271us) client_negotiation.cc:769] Sending connection context
0806 16:03:03.894586 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 16:03:03.894817 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":137,"threads_started":1}
I20260806 16:03:03.896701 11433 heartbeater.cc:347] Connected to a master server at 127.25.31.190:35505
I20260806 16:03:03.896934 11433 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:03.897397 11433 heartbeater.cc:511] Master 127.25.31.190:35505 requested a full tablet report, sending...
I20260806 16:03:03.898476 10967 ts_manager.cc:194] Registered new tserver with Master: 19ac0ed05a8045d48b8ebf8e8c836e2c (127.25.31.131:35445)
I20260806 16:03:03.899197 10967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:41723
I20260806 16:03:03.899339 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:03:03.907996 11006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:03.901412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33806 (local address 127.25.31.190:35505)
0806 16:03:03.901532 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:03.901535 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:03.901745 (+ 210us) server_negotiation.cc:407] Connection header received
0806 16:03:03.901836 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:03.901838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:03.901873 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:03.901936 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:03.902608 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.903078 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.903833 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.904033 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.904104 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:03.905253 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:03.905256 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:03.905258 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:03.905278 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:03.906712 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:03.907065 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:03.907067 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:03.907068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:03.907137 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:03.907353 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:03.907355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:03.907356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:03.907547 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:03.907645 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:03.907810 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 16:03:03.907892 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 16:03:03.910517 10967 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:03:03.913501 10967 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:03:03.928184 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:03.921420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:35445 (local address 127.0.0.1:46900)
0806 16:03:03.922262 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:03.922555 (+ 293us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:03.922687 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:03.923451 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:03.923455 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:03.923473 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:03.923631 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:03.923639 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.925047 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.925050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:03.926617 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:03.926626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.927656 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.927660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:03.927774 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:03.927816 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 16:03:03.927924 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 16:03:03.928034 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":170,"spinlock_wait_cycles":1025792,"thread_start_us":109,"threads_started":1}
I20260806 16:03:03.928356 11446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:03.922804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:35145 (local address 127.0.0.1:32856)
0806 16:03:03.923085 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:03.923099 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:03.923182 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:03.923797 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:03.923799 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:03.923821 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:03.923912 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:03.923917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.925249 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.925252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:03.926663 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:03.926675 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.927805 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.927813 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:03.927900 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:03.927922 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 16:03:03.927999 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 16:03:03.928215 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":67,"thread_start_us":132,"threads_started":1}
I20260806 16:03:03.928553 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:03.922971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32856 (local address 127.25.31.129:35145)
0806 16:03:03.923349 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:03.923367 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:03.923392 (+ 25us) server_negotiation.cc:407] Connection header received
0806 16:03:03.923490 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:03.923495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:03.923669 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:03.923825 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:03.924032 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.925104 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.926809 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.927689 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.927804 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:03.927942 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:03.928035 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 16:03:03.928131 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":85,"threads_started":1}
I20260806 16:03:03.928747 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:03.921584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46900 (local address 127.25.31.131:35445)
0806 16:03:03.921995 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:03.922008 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:03.922835 (+ 827us) server_negotiation.cc:407] Connection header received
0806 16:03:03.922918 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:03.922923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:03.923161 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:03.923306 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:03.923770 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.924901 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.926736 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.927300 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.927439 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:03.927585 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:03.928491 (+ 906us) server_negotiation.cc:299] Negotiation successful
0806 16:03:03.928600 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":160,"threads_started":1}
I20260806 16:03:03.929630 11449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:03.923688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:40263 (local address 127.0.0.1:46322)
0806 16:03:03.923995 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:03.924009 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:03.924104 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:03.924517 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:03.924520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:03.924540 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:03.924623 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:03.924628 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.926129 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.926135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:03.927497 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:03.927508 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.929240 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.929244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:03.929331 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:03.929354 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 16:03:03.929431 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 16:03:03.929505 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":17,"thread_start_us":87,"threads_started":1}
I20260806 16:03:03.929698 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:03.923330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46322 (local address 127.25.31.130:40263)
0806 16:03:03.923706 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:03.923720 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:03.924088 (+ 368us) server_negotiation.cc:407] Connection header received
0806 16:03:03.924207 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:03.924212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:03.924372 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:03.924521 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:03.924716 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.926002 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.927629 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:03.929111 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:03.929219 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:03.929339 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:03.929473 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 16:03:03.929565 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":140,"threads_started":1}
I20260806 16:03:03.931447 11092 tablet_service.cc:1467] Processing CreateTablet for tablet f9181e7d8ebb49a29c1bacd15683d7a6 (DEFAULT_TABLE table=test-table [id=42291e41e75a403e96c29f3bb4ead0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:03.931545 11366 tablet_service.cc:1467] Processing CreateTablet for tablet f9181e7d8ebb49a29c1bacd15683d7a6 (DEFAULT_TABLE table=test-table [id=42291e41e75a403e96c29f3bb4ead0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:03.932432 11092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9181e7d8ebb49a29c1bacd15683d7a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:03.932554 11366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9181e7d8ebb49a29c1bacd15683d7a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:03.932284 11228 tablet_service.cc:1467] Processing CreateTablet for tablet f9181e7d8ebb49a29c1bacd15683d7a6 (DEFAULT_TABLE table=test-table [id=42291e41e75a403e96c29f3bb4ead0fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:03.933151 11228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9181e7d8ebb49a29c1bacd15683d7a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:03.938226 11450 tablet_bootstrap.cc:492] T f9181e7d8ebb49a29c1bacd15683d7a6 P 3a0e4e7156e34c219bc7f4101edb02c0: Bootstrap starting.
I20260806 16:03:03.941010 11450 tablet_bootstrap.cc:654] T f9181e7d8ebb49a29c1bacd15683d7a6 P 3a0e4e7156e34c219bc7f4101edb02c0: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:03.942265 11450 log.cc:824] T f9181e7d8ebb49a29c1bacd15683d7a6 P 3a0e4e7156e34c219bc7f4101edb02c0: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:03.942947 11452 tablet_bootstrap.cc:492] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82: Bootstrap starting.
I20260806 16:03:03.944514 11450 tablet_bootstrap.cc:492] T f9181e7d8ebb49a29c1bacd15683d7a6 P 3a0e4e7156e34c219bc7f4101edb02c0: No bootstrap required, opened a new log
I20260806 16:03:03.944710 11450 ts_tablet_manager.cc:1397] T f9181e7d8ebb49a29c1bacd15683d7a6 P 3a0e4e7156e34c219bc7f4101edb02c0: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 16:03:03.945207 11452 tablet_bootstrap.cc:654] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:03.946519 11451 tablet_bootstrap.cc:492] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c: Bootstrap starting.
I20260806 16:03:03.946527 11452 log.cc:824] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:03.948380 11451 tablet_bootstrap.cc:654] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:03.948536 11452 tablet_bootstrap.cc:492] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82: No bootstrap required, opened a new log
I20260806 16:03:03.948716 11452 ts_tablet_manager.cc:1397] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 16:03:03.948752 11450 raft_consensus.cc:348] T f9181e7d8ebb49a29c1bacd15683d7a6 P 3a0e4e7156e34c219bc7f4101edb02c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35445 } } peers { permanent_uuid: "3a0e4e7156e34c219bc7f4101edb02c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 35145 } } peers { permanent_uuid: "9ae6440f099445f8b064d75e6e7d4b82" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 40263 } }
I20260806 16:03:03.949033 11450 raft_consensus.cc:374] T f9181e7d8ebb49a29c1bacd15683d7a6 P 3a0e4e7156e34c219bc7f4101edb02c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:03.949129 11450 raft_consensus.cc:729] T f9181e7d8ebb49a29c1bacd15683d7a6 P 3a0e4e7156e34c219bc7f4101edb02c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a0e4e7156e34c219bc7f4101edb02c0, State: Initialized, Role: FOLLOWER
I20260806 16:03:03.949591 11450 consensus_queue.cc:260] T f9181e7d8ebb49a29c1bacd15683d7a6 P 3a0e4e7156e34c219bc7f4101edb02c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35445 } } peers { permanent_uuid: "3a0e4e7156e34c219bc7f4101edb02c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 35145 } } peers { permanent_uuid: "9ae6440f099445f8b064d75e6e7d4b82" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 40263 } }
I20260806 16:03:03.949679 11451 log.cc:824] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:03.950718 11450 ts_tablet_manager.cc:1428] T f9181e7d8ebb49a29c1bacd15683d7a6 P 3a0e4e7156e34c219bc7f4101edb02c0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 16:03:03.951100 11159 heartbeater.cc:503] Master 127.25.31.190:35505 was elected leader, sending a full tablet report...
I20260806 16:03:03.951647 11451 tablet_bootstrap.cc:492] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c: No bootstrap required, opened a new log
I20260806 16:03:03.951808 11451 ts_tablet_manager.cc:1397] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260806 16:03:03.952348 11452 raft_consensus.cc:348] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35445 } } peers { permanent_uuid: "3a0e4e7156e34c219bc7f4101edb02c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 35145 } } peers { permanent_uuid: "9ae6440f099445f8b064d75e6e7d4b82" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 40263 } }
I20260806 16:03:03.952586 11452 raft_consensus.cc:374] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:03.952651 11452 raft_consensus.cc:729] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ae6440f099445f8b064d75e6e7d4b82, State: Initialized, Role: FOLLOWER
I20260806 16:03:03.953048 11452 consensus_queue.cc:260] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35445 } } peers { permanent_uuid: "3a0e4e7156e34c219bc7f4101edb02c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 35145 } } peers { permanent_uuid: "9ae6440f099445f8b064d75e6e7d4b82" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 40263 } }
I20260806 16:03:03.953634 11295 heartbeater.cc:503] Master 127.25.31.190:35505 was elected leader, sending a full tablet report...
I20260806 16:03:03.953955 11452 ts_tablet_manager.cc:1428] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 16:03:03.955756 11451 raft_consensus.cc:348] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35445 } } peers { permanent_uuid: "3a0e4e7156e34c219bc7f4101edb02c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 35145 } } peers { permanent_uuid: "9ae6440f099445f8b064d75e6e7d4b82" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 40263 } }
I20260806 16:03:03.955996 11451 raft_consensus.cc:374] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:03.956082 11451 raft_consensus.cc:729] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19ac0ed05a8045d48b8ebf8e8c836e2c, State: Initialized, Role: FOLLOWER
I20260806 16:03:03.956521 11451 consensus_queue.cc:260] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35445 } } peers { permanent_uuid: "3a0e4e7156e34c219bc7f4101edb02c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 35145 } } peers { permanent_uuid: "9ae6440f099445f8b064d75e6e7d4b82" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 40263 } }
I20260806 16:03:03.957338 11433 heartbeater.cc:503] Master 127.25.31.190:35505 was elected leader, sending a full tablet report...
I20260806 16:03:03.957378 11451 ts_tablet_manager.cc:1428] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260806 16:03:03.960044 11160 tablet.cc:2369] T f9181e7d8ebb49a29c1bacd15683d7a6 P 3a0e4e7156e34c219bc7f4101edb02c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:03:04.025394 11458 raft_consensus.cc:482] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:03:04.025591 11458 raft_consensus.cc:504] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35445 } } peers { permanent_uuid: "3a0e4e7156e34c219bc7f4101edb02c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 35145 } } peers { permanent_uuid: "9ae6440f099445f8b064d75e6e7d4b82" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 40263 } }
I20260806 16:03:04.026842 11458 leader_election.cc:288] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19ac0ed05a8045d48b8ebf8e8c836e2c (127.25.31.131:35445), 3a0e4e7156e34c219bc7f4101edb02c0 (127.25.31.129:35145)
I20260806 16:03:04.031108 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.027283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:47705 (local address 127.25.31.131:35445)
0806 16:03:04.027516 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:04.027520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:04.027536 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:03:04.027595 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:04.027599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:04.027660 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:04.027754 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:04.028131 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.028890 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.030047 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.030546 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.030679 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:04.030735 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:04.030923 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 16:03:04.030999 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260806 16:03:04.031108 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.027121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:35445 (local address 127.25.31.130:47705)
0806 16:03:04.027266 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:04.027282 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:04.027391 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:04.027876 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:04.027878 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:04.027895 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:04.028006 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:04.028010 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.029003 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.029007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:04.029916 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:04.029926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.030669 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.030673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:04.030780 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:04.030801 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 16:03:04.030898 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 16:03:04.030974 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 16:03:04.031538 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.027688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:58539 (local address 127.25.31.129:35145)
0806 16:03:04.027815 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:04.027819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:04.028133 (+ 314us) server_negotiation.cc:407] Connection header received
0806 16:03:04.028251 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:04.028253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:04.028298 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:04.028360 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:04.028644 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.029240 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.030532 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.030943 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.031077 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:04.031126 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:04.031351 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 16:03:04.031429 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 16:03:04.031744 11386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9181e7d8ebb49a29c1bacd15683d7a6" candidate_uuid: "9ae6440f099445f8b064d75e6e7d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c" is_pre_election: true
I20260806 16:03:04.031908 11460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.027567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:35145 (local address 127.25.31.130:58539)
0806 16:03:04.028032 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:04.028045 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:04.028167 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:04.028397 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:04.028400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:04.028419 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:04.028537 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:04.028543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.029358 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.029361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:04.030410 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:04.030420 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.031072 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.031075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:04.031178 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:04.031210 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 16:03:04.031710 (+ 500us) client_negotiation.cc:240] Negotiation successful
0806 16:03:04.031774 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":157,"threads_started":1}
I20260806 16:03:04.032073 11386 raft_consensus.cc:2454] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ae6440f099445f8b064d75e6e7d4b82 in term 0.
I20260806 16:03:04.032572 11183 leader_election.cc:302] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19ac0ed05a8045d48b8ebf8e8c836e2c, 9ae6440f099445f8b064d75e6e7d4b82; no voters:
I20260806 16:03:04.032581 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9181e7d8ebb49a29c1bacd15683d7a6" candidate_uuid: "9ae6440f099445f8b064d75e6e7d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a0e4e7156e34c219bc7f4101edb02c0" is_pre_election: true
I20260806 16:03:04.032843 11112 raft_consensus.cc:2454] T f9181e7d8ebb49a29c1bacd15683d7a6 P 3a0e4e7156e34c219bc7f4101edb02c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ae6440f099445f8b064d75e6e7d4b82 in term 0.
I20260806 16:03:04.032846 11458 raft_consensus.cc:2781] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:03:04.032953 11458 raft_consensus.cc:482] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:03:04.033013 11458 raft_consensus.cc:3037] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:04.034109 11458 raft_consensus.cc:504] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35445 } } peers { permanent_uuid: "3a0e4e7156e34c219bc7f4101edb02c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 35145 } } peers { permanent_uuid: "9ae6440f099445f8b064d75e6e7d4b82" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 40263 } }
I20260806 16:03:04.034472 11458 leader_election.cc:288] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [CANDIDATE]: Term 1 election: Requested vote from peers 19ac0ed05a8045d48b8ebf8e8c836e2c (127.25.31.131:35445), 3a0e4e7156e34c219bc7f4101edb02c0 (127.25.31.129:35145)
I20260806 16:03:04.034839 11386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9181e7d8ebb49a29c1bacd15683d7a6" candidate_uuid: "9ae6440f099445f8b064d75e6e7d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c"
I20260806 16:03:04.034986 11386 raft_consensus.cc:3037] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:04.035010 11112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9181e7d8ebb49a29c1bacd15683d7a6" candidate_uuid: "9ae6440f099445f8b064d75e6e7d4b82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a0e4e7156e34c219bc7f4101edb02c0"
I20260806 16:03:04.035141 11112 raft_consensus.cc:3037] T f9181e7d8ebb49a29c1bacd15683d7a6 P 3a0e4e7156e34c219bc7f4101edb02c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:04.036041 11386 raft_consensus.cc:2454] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ae6440f099445f8b064d75e6e7d4b82 in term 1.
I20260806 16:03:04.036231 11112 raft_consensus.cc:2454] T f9181e7d8ebb49a29c1bacd15683d7a6 P 3a0e4e7156e34c219bc7f4101edb02c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ae6440f099445f8b064d75e6e7d4b82 in term 1.
I20260806 16:03:04.036356 11183 leader_election.cc:302] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19ac0ed05a8045d48b8ebf8e8c836e2c, 9ae6440f099445f8b064d75e6e7d4b82; no voters:
I20260806 16:03:04.036633 11458 raft_consensus.cc:2781] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:04.036877 11458 raft_consensus.cc:686] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [term 1 LEADER]: Becoming Leader. State: Replica: 9ae6440f099445f8b064d75e6e7d4b82, State: Running, Role: LEADER
I20260806 16:03:04.037211 11458 consensus_queue.cc:237] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35445 } } peers { permanent_uuid: "3a0e4e7156e34c219bc7f4101edb02c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 35145 } } peers { permanent_uuid: "9ae6440f099445f8b064d75e6e7d4b82" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 40263 } }
I20260806 16:03:04.040349 10966 catalog_manager.cc:5495] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ae6440f099445f8b064d75e6e7d4b82 (127.25.31.130). New cstate: current_term: 1 leader_uuid: "9ae6440f099445f8b064d75e6e7d4b82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a0e4e7156e34c219bc7f4101edb02c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ae6440f099445f8b064d75e6e7d4b82" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 40263 } health_report { overall_health: HEALTHY } } }
I20260806 16:03:04.054962 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.051283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46324 (local address 127.25.31.130:40263)
0806 16:03:04.051422 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:04.051428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:04.051448 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:03:04.051643 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:04.051646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:04.051697 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:04.051804 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:04.052057 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.052653 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.053522 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.053726 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.053832 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:04.054132 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:04.054573 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:04.054698 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:04.054755 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 16:03:04.054804 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 16:03:04.060410 11112 raft_consensus.cc:1261] T f9181e7d8ebb49a29c1bacd15683d7a6 P 3a0e4e7156e34c219bc7f4101edb02c0 [term 1 FOLLOWER]: Refusing update from remote peer 9ae6440f099445f8b064d75e6e7d4b82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:04.060411 11386 raft_consensus.cc:1261] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c [term 1 FOLLOWER]: Refusing update from remote peer 9ae6440f099445f8b064d75e6e7d4b82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:04.061070 11458 consensus_queue.cc:1035] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a0e4e7156e34c219bc7f4101edb02c0" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:03:04.061254 11461 consensus_queue.cc:1035] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19ac0ed05a8045d48b8ebf8e8c836e2c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:03:04.068914 11465 mvcc.cc:205] Tried to move back new op lower bound from 7315587825904799744 to 7315587825819201536. Current Snapshot: MvccSnapshot[applied={T|T < 7315587825904799744}]
I20260806 16:03:04.078099 10967 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:03:04.078246 10967 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:03:04.080646 10967 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44011db6aa87487582a3fcbb5d23cc8b: Sending DeleteTablet for 3 replicas of tablet f9181e7d8ebb49a29c1bacd15683d7a6
I20260806 16:03:04.081501 11228 tablet_service.cc:1514] Processing DeleteTablet for tablet f9181e7d8ebb49a29c1bacd15683d7a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:04 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:46322
I20260806 16:03:04.081657 11092 tablet_service.cc:1514] Processing DeleteTablet for tablet f9181e7d8ebb49a29c1bacd15683d7a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:04 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:32856
I20260806 16:03:04.081964 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 11029
I20260806 16:03:04.082115 11366 tablet_service.cc:1514] Processing DeleteTablet for tablet f9181e7d8ebb49a29c1bacd15683d7a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:04 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:46900
I20260806 16:03:04.083101 11474 tablet_replica.cc:330] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82: stopping tablet replica
I20260806 16:03:04.083462 11474 raft_consensus.cc:2229] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:03:04.083894 11474 raft_consensus.cc:2258] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:03:04.084826 11476 tablet_replica.cc:330] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c: stopping tablet replica
I20260806 16:03:04.085105 11476 raft_consensus.cc:2229] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:03:04.085435 11476 raft_consensus.cc:2258] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:03:04.086979 11474 ts_tablet_manager.cc:1905] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 16:03:04.089422 10953 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:35145 (error 108)
I20260806 16:03:04.089908 11474 ts_tablet_manager.cc:1918] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:03:04.090008 11474 log.cc:1196] T f9181e7d8ebb49a29c1bacd15683d7a6 P 9ae6440f099445f8b064d75e6e7d4b82: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/f9181e7d8ebb49a29c1bacd15683d7a6
I20260806 16:03:04.090453 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.090149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:35145 (local address 127.0.0.1:32872)
0806 16:03:04.090253 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:04.090348 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:35145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 16:03:04.090574 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 11164
W20260806 16:03:04.091300 10953 catalog_manager.cc:4584] TS 3a0e4e7156e34c219bc7f4101edb02c0 (127.25.31.129:35145): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f9181e7d8ebb49a29c1bacd15683d7a6: Network error: Client connection negotiation failed: client connection to 127.25.31.129:35145: connect: Connection refused (error 111)
I20260806 16:03:04.094880 11476 ts_tablet_manager.cc:1905] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:03:04.098335 11476 ts_tablet_manager.cc:1918] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:03:04.098444 11476 log.cc:1196] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/f9181e7d8ebb49a29c1bacd15683d7a6
I20260806 16:03:04.098802 11476 ts_tablet_manager.cc:1939] T f9181e7d8ebb49a29c1bacd15683d7a6 P 19ac0ed05a8045d48b8ebf8e8c836e2c: Deleting consensus metadata
I20260806 16:03:04.100150 10953 catalog_manager.cc:4841] TS 19ac0ed05a8045d48b8ebf8e8c836e2c (127.25.31.131:35445): tablet f9181e7d8ebb49a29c1bacd15683d7a6 (table test-table [id=42291e41e75a403e96c29f3bb4ead0fa]) successfully deleted
I20260806 16:03:04.100553 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 11300
I20260806 16:03:04.101521 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.100786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:40263 (local address 127.0.0.1:46328)
0806 16:03:04.101324 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:04.101399 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.130:40263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432}
I20260806 16:03:04.106993 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 10934
2026-08-06T16:03:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2296 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T16:03:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:04Z Disabled control of system clock
I20260806 16:03:04.138784 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:41513
--webserver_interface=127.25.31.190
--webserver_port=0
--builtin_ntp_servers=127.25.31.148:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:41513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:03:04.247594 11481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:04.247843 11481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:04.247879 11481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:04.251768 11481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:04.251820 11481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:04.251835 11481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:04.251852 11481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:04.251868 11481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:04.256597 11481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:41513
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:41513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:04.257576 11481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:04.258950 11481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:04.267078 11486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:04.267051 11487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:04.267267 11481 server_base.cc:1034] running on GCE node
W20260806 16:03:04.267059 11489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:04.267925 11481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:04.269004 11481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:04.270192 11481 hybrid_clock.cc:648] HybridClock initialized: now 1786032184270176 us; error 28 us; skew 500 ppm
I20260806 16:03:04.276857 11481 webserver.cc:460] Webserver started at http://127.25.31.190:46689/ using document root <none> and password file <none>
I20260806 16:03:04.277446 11481 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:04.277540 11481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:04.277778 11481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:04.279861 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "6bfadf058f204b4ea2bfdae6a55241f4"
format_stamp: "Formatted at 2026-08-06 16:03:04 on dist-test-slave-tm1g"
server_key: "7935bd20514d7f29eeeef3542c24e2f4"
server_key_iv: "f574bc086f84342197c6e7d1509ac9c2"
server_key_version: "encryptionkey@0"
I20260806 16:03:04.280436 11481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "6bfadf058f204b4ea2bfdae6a55241f4"
format_stamp: "Formatted at 2026-08-06 16:03:04 on dist-test-slave-tm1g"
server_key: "7935bd20514d7f29eeeef3542c24e2f4"
server_key_iv: "f574bc086f84342197c6e7d1509ac9c2"
server_key_version: "encryptionkey@0"
I20260806 16:03:04.284158 11481 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260806 16:03:04.286568 11495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:04.287720 11481 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 16:03:04.287858 11481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "6bfadf058f204b4ea2bfdae6a55241f4"
format_stamp: "Formatted at 2026-08-06 16:03:04 on dist-test-slave-tm1g"
server_key: "7935bd20514d7f29eeeef3542c24e2f4"
server_key_iv: "f574bc086f84342197c6e7d1509ac9c2"
server_key_version: "encryptionkey@0"
I20260806 16:03:04.287969 11481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:04.299144 11481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:04.299760 11481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:04.299955 11481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:04.306640 11481 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:41513
I20260806 16:03:04.306670 11547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:41513 every 8 connection(s)
I20260806 16:03:04.307539 11481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:04.310571 11549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:04.314208 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 11481
I20260806 16:03:04.314368 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:04.314664 25726 external_mini_cluster.cc:1428] Setting key 531f970a7b675503c4c4d97e060ec8de
I20260806 16:03:04.316181 11549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4: Bootstrap starting.
I20260806 16:03:04.318462 11549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:04.319209 11549 log.cc:824] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:04.320956 11549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4: No bootstrap required, opened a new log
I20260806 16:03:04.323580 11552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.316091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48614 (local address 127.25.31.190:41513)
0806 16:03:04.316713 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:04.316728 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:04.316762 (+ 34us) server_negotiation.cc:407] Connection header received
0806 16:03:04.317363 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:04.317378 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:04.317764 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:04.318030 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:04.318420 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.319487 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.320384 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.320856 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.321044 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:04.321915 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:04.321920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:04.321934 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:04.322342 (+ 408us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:04.322413 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:04.322440 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:04.322546 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:04.322667 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 16:03:04.322911 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":156,"threads_started":1}
I20260806 16:03:04.324687 11549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bfadf058f204b4ea2bfdae6a55241f4" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41513 } }
I20260806 16:03:04.324941 11549 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:04.325009 11549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bfadf058f204b4ea2bfdae6a55241f4, State: Initialized, Role: FOLLOWER
I20260806 16:03:04.325459 11549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bfadf058f204b4ea2bfdae6a55241f4" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41513 } }
I20260806 16:03:04.325645 11549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:04.325717 11549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:04.325816 11549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:04.326692 11549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bfadf058f204b4ea2bfdae6a55241f4" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41513 } }
I20260806 16:03:04.326972 11549 leader_election.cc:302] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bfadf058f204b4ea2bfdae6a55241f4; no voters:
I20260806 16:03:04.327201 11549 leader_election.cc:288] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:04.327340 11554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:04.327570 11554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [term 1 LEADER]: Becoming Leader. State: Replica: 6bfadf058f204b4ea2bfdae6a55241f4, State: Running, Role: LEADER
I20260806 16:03:04.327896 11554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bfadf058f204b4ea2bfdae6a55241f4" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41513 } }
I20260806 16:03:04.328382 11549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:04.330232 11555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bfadf058f204b4ea2bfdae6a55241f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bfadf058f204b4ea2bfdae6a55241f4" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41513 } } }
I20260806 16:03:04.330377 11555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:04.330392 11556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bfadf058f204b4ea2bfdae6a55241f4. Latest consensus state: current_term: 1 leader_uuid: "6bfadf058f204b4ea2bfdae6a55241f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bfadf058f204b4ea2bfdae6a55241f4" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41513 } } }
I20260806 16:03:04.330744 11556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4 [sys.catalog]: This master's current role is: LEADER
W20260806 16:03:04.333280 11571 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:03:04.333377 11571 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:03:04.333488 11569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:04.334201 11569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:04.339171 11569 catalog_manager.cc:1347] Generated new cluster ID: bd36febbab5441019e5d56ed57de4ac2
I20260806 16:03:04.339232 11569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:04.356813 11569 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:04.358032 11569 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:04.368376 11569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4: Generated new TSK 0
I20260806 16:03:04.369071 11569 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:04.378762 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:41513
--builtin_ntp_servers=127.25.31.148:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:03:04.506336 11574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:04.506620 11574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:04.506692 11574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:04.510666 11574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:04.510735 11574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:04.510826 11574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:03:04.515450 11574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:41513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:04.516467 11574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:04.517800 11574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:04.526963 11582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:04.526964 11580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:04.526993 11579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:04.527174 11574 server_base.cc:1034] running on GCE node
I20260806 16:03:04.527717 11574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:04.528391 11574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:04.529577 11574 hybrid_clock.cc:648] HybridClock initialized: now 1786032184529568 us; error 28 us; skew 500 ppm
I20260806 16:03:04.531404 11574 webserver.cc:460] Webserver started at http://127.25.31.129:40999/ using document root <none> and password file <none>
I20260806 16:03:04.531967 11574 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:04.532060 11574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:04.532291 11574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:04.534248 11574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "234b4e4d198c4a7a97900a4e542547ce"
format_stamp: "Formatted at 2026-08-06 16:03:04 on dist-test-slave-tm1g"
server_key: "b5a205bad46b72480dd40404459c74b4"
server_key_iv: "2d0d9f04bc19bcd930e8a42cb6e04de0"
server_key_version: "encryptionkey@0"
I20260806 16:03:04.534858 11574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "234b4e4d198c4a7a97900a4e542547ce"
format_stamp: "Formatted at 2026-08-06 16:03:04 on dist-test-slave-tm1g"
server_key: "b5a205bad46b72480dd40404459c74b4"
server_key_iv: "2d0d9f04bc19bcd930e8a42cb6e04de0"
server_key_version: "encryptionkey@0"
I20260806 16:03:04.538476 11574 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 16:03:04.540714 11588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:04.541874 11574 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:03:04.542014 11574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "234b4e4d198c4a7a97900a4e542547ce"
format_stamp: "Formatted at 2026-08-06 16:03:04 on dist-test-slave-tm1g"
server_key: "b5a205bad46b72480dd40404459c74b4"
server_key_iv: "2d0d9f04bc19bcd930e8a42cb6e04de0"
server_key_version: "encryptionkey@0"
I20260806 16:03:04.542125 11574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:04.563668 11574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:04.564321 11574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:04.564528 11574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:04.565093 11574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:04.566097 11574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:04.566164 11574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:04.566241 11574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:04.566282 11574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:04.574900 11574 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:43673
I20260806 16:03:04.574930 11704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:43673 every 8 connection(s)
I20260806 16:03:04.575866 11574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 16:03:04.584892 11552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.577610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:58803 (local address 127.25.31.190:41513)
0806 16:03:04.577754 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:04.577759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:04.578797 (+ 1038us) server_negotiation.cc:407] Connection header received
0806 16:03:04.580191 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:04.580195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:04.580255 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:04.580357 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:04.581206 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.581750 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.582978 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.583203 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.583318 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:04.584407 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:04.584410 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:04.584412 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:04.584422 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:04.584441 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:04.584448 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:04.584529 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:04.584683 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 16:03:04.584747 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 16:03:04.585704 11707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.577973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:41513 (local address 127.25.31.129:58803)
0806 16:03:04.578651 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:04.578691 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:04.579982 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:04.580526 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:04.580533 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:04.580614 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:04.581066 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:04.581078 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.581880 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.581888 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:04.582841 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:04.582851 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.583357 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.583361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:04.583527 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:04.584154 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:03:04.584155 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:04.584192 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:03:04.584251 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:04.584530 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:04.584542 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:04.584552 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 16:03:04.584681 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 16:03:04.584940 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":117,"threads_started":1}
I20260806 16:03:04.586259 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 11574
I20260806 16:03:04.586404 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 16:03:04.586740 25726 external_mini_cluster.cc:1428] Setting key 9f882f90fe41586227fe2e2e6fb65e9e
I20260806 16:03:04.587200 11705 heartbeater.cc:347] Connected to a master server at 127.25.31.190:41513
I20260806 16:03:04.587435 11705 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:04.587963 11705 heartbeater.cc:511] Master 127.25.31.190:41513 requested a full tablet report, sending...
I20260806 16:03:04.588809 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:41513
--builtin_ntp_servers=127.25.31.148:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:03:04.589542 11512 ts_manager.cc:194] Registered new tserver with Master: 234b4e4d198c4a7a97900a4e542547ce (127.25.31.129:43673)
I20260806 16:03:04.590972 11512 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.31.129:58803
W20260806 16:03:04.696527 11708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:04.696799 11708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:04.696864 11708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:04.700771 11708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:04.700850 11708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:04.700945 11708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:03:04.705540 11708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:41513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:04.706549 11708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:04.708067 11708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:04.717384 11716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:04.717338 11714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:04.717443 11713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:04.717717 11708 server_base.cc:1034] running on GCE node
I20260806 16:03:04.718091 11708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:04.718712 11708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:04.719875 11708 hybrid_clock.cc:648] HybridClock initialized: now 1786032184719862 us; error 24 us; skew 500 ppm
I20260806 16:03:04.721621 11708 webserver.cc:460] Webserver started at http://127.25.31.130:40447/ using document root <none> and password file <none>
I20260806 16:03:04.722167 11708 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:04.722229 11708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:04.722462 11708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:04.724408 11708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "632b24d58d084af3a94c8270260024cd"
format_stamp: "Formatted at 2026-08-06 16:03:04 on dist-test-slave-tm1g"
server_key: "d36abc030bbc943488d0eb748e9706b0"
server_key_iv: "1b097cf83ce66b10030aa1210f3c3129"
server_key_version: "encryptionkey@0"
I20260806 16:03:04.724984 11708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "632b24d58d084af3a94c8270260024cd"
format_stamp: "Formatted at 2026-08-06 16:03:04 on dist-test-slave-tm1g"
server_key: "d36abc030bbc943488d0eb748e9706b0"
server_key_iv: "1b097cf83ce66b10030aa1210f3c3129"
server_key_version: "encryptionkey@0"
I20260806 16:03:04.728583 11708 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 16:03:04.731060 11722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:04.732255 11708 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 16:03:04.732385 11708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "632b24d58d084af3a94c8270260024cd"
format_stamp: "Formatted at 2026-08-06 16:03:04 on dist-test-slave-tm1g"
server_key: "d36abc030bbc943488d0eb748e9706b0"
server_key_iv: "1b097cf83ce66b10030aa1210f3c3129"
server_key_version: "encryptionkey@0"
I20260806 16:03:04.732491 11708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:04.758141 11708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:04.758852 11708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:04.759060 11708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:04.759652 11708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:04.760586 11708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:04.760632 11708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:04.760701 11708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:04.760746 11708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:04.769441 11708 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:36203
I20260806 16:03:04.769480 11837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:36203 every 8 connection(s)
I20260806 16:03:04.770375 11708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:03:04.775741 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 11708
I20260806 16:03:04.775880 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 16:03:04.776165 25726 external_mini_cluster.cc:1428] Setting key f94096292196be1ea2fac15ea4bd2c9a
I20260806 16:03:04.778170 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:41513
--builtin_ntp_servers=127.25.31.148:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:03:04.779881 11552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.772130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:60241 (local address 127.25.31.190:41513)
0806 16:03:04.772280 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:04.772284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:04.773342 (+ 1058us) server_negotiation.cc:407] Connection header received
0806 16:03:04.774669 (+ 1327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:04.774671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:04.774715 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:04.774787 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:04.775712 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.776340 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.777777 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.777994 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.778074 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:04.779343 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:04.779345 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:04.779347 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:04.779358 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:04.779377 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:04.779384 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:04.779443 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:04.779625 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 16:03:04.779674 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:03:04.780704 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.772569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:41513 (local address 127.25.31.130:60241)
0806 16:03:04.773200 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:04.773238 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:04.774496 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:04.774929 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:04.774936 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:04.775013 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:04.775565 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:04.775586 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.776471 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.776474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:04.777656 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:04.777668 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.778150 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.778155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:04.778416 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:04.779050 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:03:04.779054 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:04.779108 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:03:04.779191 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:04.779473 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:04.779485 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:04.779494 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 16:03:04.779620 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 16:03:04.779936 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":119,"threads_started":1}
I20260806 16:03:04.781848 11838 heartbeater.cc:347] Connected to a master server at 127.25.31.190:41513
I20260806 16:03:04.782080 11838 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:04.782637 11838 heartbeater.cc:511] Master 127.25.31.190:41513 requested a full tablet report, sending...
I20260806 16:03:04.783643 11512 ts_manager.cc:194] Registered new tserver with Master: 632b24d58d084af3a94c8270260024cd (127.25.31.130:36203)
I20260806 16:03:04.784198 11512 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.31.130:60241
W20260806 16:03:04.888674 11841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:04.888896 11841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:04.888931 11841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:04.893102 11841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:04.893153 11841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:04.893234 11841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:03:04.897881 11841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:41513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:04.898842 11841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:04.900132 11841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:04.909583 11848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:04.909606 11850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:04.909663 11841 server_base.cc:1034] running on GCE node
W20260806 16:03:04.909602 11847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:04.910202 11841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:04.910808 11841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:04.911968 11841 hybrid_clock.cc:648] HybridClock initialized: now 1786032184911955 us; error 24 us; skew 500 ppm
I20260806 16:03:04.913682 11841 webserver.cc:460] Webserver started at http://127.25.31.131:37261/ using document root <none> and password file <none>
I20260806 16:03:04.914224 11841 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:04.914284 11841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:04.914526 11841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:04.916435 11841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "4af0cb422e2f4ae99ff96147a838281c"
format_stamp: "Formatted at 2026-08-06 16:03:04 on dist-test-slave-tm1g"
server_key: "b8b24b7e7aee297a62d63676869bcf25"
server_key_iv: "e92330ee32f68efda390b9edeeaf9d23"
server_key_version: "encryptionkey@0"
I20260806 16:03:04.917006 11841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "4af0cb422e2f4ae99ff96147a838281c"
format_stamp: "Formatted at 2026-08-06 16:03:04 on dist-test-slave-tm1g"
server_key: "b8b24b7e7aee297a62d63676869bcf25"
server_key_iv: "e92330ee32f68efda390b9edeeaf9d23"
server_key_version: "encryptionkey@0"
I20260806 16:03:04.920488 11841 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 16:03:04.922737 11856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:04.923862 11841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:03:04.924000 11841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "4af0cb422e2f4ae99ff96147a838281c"
format_stamp: "Formatted at 2026-08-06 16:03:04 on dist-test-slave-tm1g"
server_key: "b8b24b7e7aee297a62d63676869bcf25"
server_key_iv: "e92330ee32f68efda390b9edeeaf9d23"
server_key_version: "encryptionkey@0"
I20260806 16:03:04.924108 11841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:04.943485 11841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:04.944106 11841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:04.944296 11841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:04.944854 11841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:04.945844 11841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:04.945915 11841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:04.945981 11841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:04.946033 11841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:04.954530 11841 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:37379
I20260806 16:03:04.954563 11970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:37379 every 8 connection(s)
I20260806 16:03:04.955384 11841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:03:04.963954 11552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.957125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:42609 (local address 127.25.31.190:41513)
0806 16:03:04.957271 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:04.957276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:04.958292 (+ 1016us) server_negotiation.cc:407] Connection header received
0806 16:03:04.959618 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:04.959621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:04.959666 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:04.959740 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:04.960559 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.961068 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.962182 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.962453 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.962538 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:04.963566 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:04.963569 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:04.963571 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:04.963581 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:04.963596 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:04.963603 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:04.963654 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:04.963785 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 16:03:04.963832 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 16:03:04.964898 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.957553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:41513 (local address 127.25.31.131:42609)
0806 16:03:04.958157 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:04.958195 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:04.959443 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:04.959899 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:04.959906 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:04.959983 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:04.960439 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:04.960451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.961188 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.961191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:04.962078 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:04.962087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.962586 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.962589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:04.962720 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:04.963339 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:03:04.963341 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:04.963379 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:03:04.963443 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:04.963672 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:04.963679 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:04.963683 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 16:03:04.963771 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 16:03:04.964033 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":106,"threads_started":1}
I20260806 16:03:04.965020 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 11841
I20260806 16:03:04.965129 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 16:03:04.965466 25726 external_mini_cluster.cc:1428] Setting key 9298615450c4035048fc1c5cacb1e50f
I20260806 16:03:04.966089 11971 heartbeater.cc:347] Connected to a master server at 127.25.31.190:41513
I20260806 16:03:04.966403 11971 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:04.967011 11971 heartbeater.cc:511] Master 127.25.31.190:41513 requested a full tablet report, sending...
I20260806 16:03:04.968078 11512 ts_manager.cc:194] Registered new tserver with Master: 4af0cb422e2f4ae99ff96147a838281c (127.25.31.131:37379)
I20260806 16:03:04.968638 11512 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.31.131:42609
I20260806 16:03:04.979225 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:03:04.984884 11552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.981109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48622 (local address 127.25.31.190:41513)
0806 16:03:04.981258 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:04.981263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:04.981422 (+ 159us) server_negotiation.cc:407] Connection header received
0806 16:03:04.981564 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:04.981568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:04.981614 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:04.981697 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:04.981984 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.982508 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.983363 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:04.983583 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:04.983689 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:04.984446 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:04.984454 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:04.984456 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:04.984473 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:04.984491 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:04.984498 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:04.984573 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:04.984687 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 16:03:04.984740 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 16:03:04.987660 11512 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:03:04.990589 11512 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:03:05.004945 11984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.998798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:36203 (local address 127.0.0.1:35676)
0806 16:03:04.999943 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:04.999957 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:05.000062 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:05.000899 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:05.000902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:05.000933 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:05.001067 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:05.001076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.002643 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.002647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:05.003773 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:05.003781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.004546 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.004550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:05.004637 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:05.004659 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 16:03:05.004720 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 16:03:05.004788 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1081,"mutex_wait_us":196,"thread_start_us":79,"threads_started":1}
I20260806 16:03:05.005118 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.999482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35676 (local address 127.25.31.130:36203)
0806 16:03:04.999888 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:04.999901 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:05.000457 (+ 556us) server_negotiation.cc:407] Connection header received
0806 16:03:05.000546 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:05.000552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:05.000741 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:05.001199 (+ 458us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:05.001350 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.002504 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.003896 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.004420 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.004547 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:05.004776 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:05.004830 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 16:03:05.004930 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":170,"threads_started":1}
I20260806 16:03:05.005409 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.998855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46106 (local address 127.25.31.131:37379)
0806 16:03:05.000339 (+ 1484us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:05.000350 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:05.000373 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:03:05.000442 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:05.000446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:05.000597 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:05.000754 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:05.001030 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.001884 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.003422 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.004224 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.004875 (+ 651us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:05.005080 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:05.005127 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 16:03:05.005236 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1373,"thread_start_us":118,"threads_started":1}
I20260806 16:03:05.005154 11987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.999929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:43673 (local address 127.0.0.1:44642)
0806 16:03:05.000316 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:05.000330 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:05.000414 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:05.001100 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:05.001103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:05.001122 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:05.001196 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:05.001201 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.002643 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.002647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:05.003715 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:05.003726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.004799 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.004803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:05.004887 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:05.004907 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 16:03:05.004956 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 16:03:05.005025 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":3,"thread_start_us":105,"threads_started":1}
I20260806 16:03:05.005820 11988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:05.000147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44642 (local address 127.25.31.129:43673)
0806 16:03:05.000672 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:05.000682 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:05.000703 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:03:05.000773 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:05.000778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:05.000941 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:05.001124 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:05.001277 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.002504 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.003873 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.004674 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.005265 (+ 591us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:05.005479 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:05.005532 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 16:03:05.005634 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":121,"threads_started":1}
I20260806 16:03:05.005498 11983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:04.998642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:37379 (local address 127.0.0.1:46106)
0806 16:03:04.999046 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:04.999095 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:04.999221 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:05.000747 (+ 1526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:05.000752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:05.000775 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:05.000911 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:05.000918 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.002008 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.002012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:05.003284 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:05.003295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.004362 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.004366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:05.004474 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:05.004524 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 16:03:05.004611 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 16:03:05.004750 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":83,"threads_started":1}
I20260806 16:03:05.008479 11637 tablet_service.cc:1467] Processing CreateTablet for tablet a650e10c73cd4c54a5a2ab2aee511aa1 (DEFAULT_TABLE table=test-table [id=e3274d8c96cd43028c3877cf3beb7543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:05.008708 11771 tablet_service.cc:1467] Processing CreateTablet for tablet a650e10c73cd4c54a5a2ab2aee511aa1 (DEFAULT_TABLE table=test-table [id=e3274d8c96cd43028c3877cf3beb7543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:05.008708 11905 tablet_service.cc:1467] Processing CreateTablet for tablet a650e10c73cd4c54a5a2ab2aee511aa1 (DEFAULT_TABLE table=test-table [id=e3274d8c96cd43028c3877cf3beb7543]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:05.009497 11637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a650e10c73cd4c54a5a2ab2aee511aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:05.009696 11771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a650e10c73cd4c54a5a2ab2aee511aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:05.009716 11905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a650e10c73cd4c54a5a2ab2aee511aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:05.016101 11989 tablet_bootstrap.cc:492] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce: Bootstrap starting.
I20260806 16:03:05.017215 11990 tablet_bootstrap.cc:492] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd: Bootstrap starting.
I20260806 16:03:05.017570 11991 tablet_bootstrap.cc:492] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c: Bootstrap starting.
I20260806 16:03:05.018586 11989 tablet_bootstrap.cc:654] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:05.019307 11991 tablet_bootstrap.cc:654] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:05.019392 11990 tablet_bootstrap.cc:654] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:05.019506 11989 log.cc:824] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:05.020172 11990 log.cc:824] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:05.020157 11991 log.cc:824] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:05.021653 11989 tablet_bootstrap.cc:492] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce: No bootstrap required, opened a new log
I20260806 16:03:05.021852 11989 ts_tablet_manager.cc:1397] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 16:03:05.022086 11991 tablet_bootstrap.cc:492] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c: No bootstrap required, opened a new log
I20260806 16:03:05.022279 11991 ts_tablet_manager.cc:1397] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 16:03:05.022535 11990 tablet_bootstrap.cc:492] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd: No bootstrap required, opened a new log
I20260806 16:03:05.022728 11990 ts_tablet_manager.cc:1397] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 16:03:05.025679 11990 raft_consensus.cc:348] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af0cb422e2f4ae99ff96147a838281c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37379 } } peers { permanent_uuid: "632b24d58d084af3a94c8270260024cd" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 36203 } } peers { permanent_uuid: "234b4e4d198c4a7a97900a4e542547ce" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43673 } }
I20260806 16:03:05.025918 11990 raft_consensus.cc:374] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:05.025981 11990 raft_consensus.cc:729] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632b24d58d084af3a94c8270260024cd, State: Initialized, Role: FOLLOWER
I20260806 16:03:05.026280 11989 raft_consensus.cc:348] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af0cb422e2f4ae99ff96147a838281c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37379 } } peers { permanent_uuid: "632b24d58d084af3a94c8270260024cd" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 36203 } } peers { permanent_uuid: "234b4e4d198c4a7a97900a4e542547ce" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43673 } }
I20260806 16:03:05.026280 11991 raft_consensus.cc:348] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af0cb422e2f4ae99ff96147a838281c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37379 } } peers { permanent_uuid: "632b24d58d084af3a94c8270260024cd" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 36203 } } peers { permanent_uuid: "234b4e4d198c4a7a97900a4e542547ce" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43673 } }
I20260806 16:03:05.026459 11990 consensus_queue.cc:260] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af0cb422e2f4ae99ff96147a838281c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37379 } } peers { permanent_uuid: "632b24d58d084af3a94c8270260024cd" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 36203 } } peers { permanent_uuid: "234b4e4d198c4a7a97900a4e542547ce" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43673 } }
I20260806 16:03:05.026558 11991 raft_consensus.cc:374] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:05.026558 11989 raft_consensus.cc:374] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:05.026612 11989 raft_consensus.cc:729] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 234b4e4d198c4a7a97900a4e542547ce, State: Initialized, Role: FOLLOWER
I20260806 16:03:05.026621 11991 raft_consensus.cc:729] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af0cb422e2f4ae99ff96147a838281c, State: Initialized, Role: FOLLOWER
I20260806 16:03:05.027040 11991 consensus_queue.cc:260] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af0cb422e2f4ae99ff96147a838281c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37379 } } peers { permanent_uuid: "632b24d58d084af3a94c8270260024cd" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 36203 } } peers { permanent_uuid: "234b4e4d198c4a7a97900a4e542547ce" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43673 } }
I20260806 16:03:05.027040 11989 consensus_queue.cc:260] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af0cb422e2f4ae99ff96147a838281c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37379 } } peers { permanent_uuid: "632b24d58d084af3a94c8270260024cd" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 36203 } } peers { permanent_uuid: "234b4e4d198c4a7a97900a4e542547ce" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43673 } }
I20260806 16:03:05.027156 11838 heartbeater.cc:503] Master 127.25.31.190:41513 was elected leader, sending a full tablet report...
I20260806 16:03:05.027505 11990 ts_tablet_manager.cc:1428] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 16:03:05.027961 11989 ts_tablet_manager.cc:1428] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260806 16:03:05.028136 11705 heartbeater.cc:503] Master 127.25.31.190:41513 was elected leader, sending a full tablet report...
I20260806 16:03:05.028008 11991 ts_tablet_manager.cc:1428] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260806 16:03:05.028407 11971 heartbeater.cc:503] Master 127.25.31.190:41513 was elected leader, sending a full tablet report...
W20260806 16:03:05.077011 11706 tablet.cc:2369] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:03:05.187891 11996 raft_consensus.cc:482] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:03:05.188118 11996 raft_consensus.cc:504] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af0cb422e2f4ae99ff96147a838281c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37379 } } peers { permanent_uuid: "632b24d58d084af3a94c8270260024cd" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 36203 } } peers { permanent_uuid: "234b4e4d198c4a7a97900a4e542547ce" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43673 } }
I20260806 16:03:05.189291 11996 leader_election.cc:288] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 632b24d58d084af3a94c8270260024cd (127.25.31.130:36203), 234b4e4d198c4a7a97900a4e542547ce (127.25.31.129:43673)
I20260806 16:03:05.193606 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:05.189550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:36203 (local address 127.25.31.131:52269)
0806 16:03:05.189693 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:05.189712 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:05.189809 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:05.190189 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:05.190195 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:05.190221 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:05.190379 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:05.190390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.191385 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.191389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:05.192442 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:05.192452 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.193139 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.193143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:05.193244 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:05.193273 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 16:03:05.193343 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 16:03:05.193429 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":49}
I20260806 16:03:05.193610 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:05.189705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:52269 (local address 127.25.31.130:36203)
0806 16:03:05.189850 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:05.189855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:05.189872 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:03:05.189940 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:05.189943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:05.189997 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:05.190102 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:05.190532 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.191281 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.192585 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.193008 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.193104 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:05.193187 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:05.193392 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 16:03:05.193481 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 16:03:05.193639 11988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:05.190025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:55881 (local address 127.25.31.129:43673)
0806 16:03:05.190173 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:05.190178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:05.190354 (+ 176us) server_negotiation.cc:407] Connection header received
0806 16:03:05.190462 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:05.190466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:05.190530 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:05.190589 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:05.190892 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.191522 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.192650 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.193039 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.193174 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:05.193226 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:05.193452 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 16:03:05.193532 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 16:03:05.193622 11998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:05.189862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:43673 (local address 127.25.31.131:55881)
0806 16:03:05.190255 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:05.190267 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:05.190378 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:05.190629 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:05.190632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:05.190649 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:05.190779 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:05.190786 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.191638 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.191642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:05.192494 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:05.192503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.193203 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.193209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:05.193300 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:05.193317 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 16:03:05.193414 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 16:03:05.193477 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":128,"threads_started":1}
I20260806 16:03:05.194331 11658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a650e10c73cd4c54a5a2ab2aee511aa1" candidate_uuid: "4af0cb422e2f4ae99ff96147a838281c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "234b4e4d198c4a7a97900a4e542547ce" is_pre_election: true
I20260806 16:03:05.194358 11792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a650e10c73cd4c54a5a2ab2aee511aa1" candidate_uuid: "4af0cb422e2f4ae99ff96147a838281c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632b24d58d084af3a94c8270260024cd" is_pre_election: true
I20260806 16:03:05.194633 11658 raft_consensus.cc:2454] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4af0cb422e2f4ae99ff96147a838281c in term 0.
I20260806 16:03:05.194636 11792 raft_consensus.cc:2454] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4af0cb422e2f4ae99ff96147a838281c in term 0.
I20260806 16:03:05.195117 11859 leader_election.cc:302] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4af0cb422e2f4ae99ff96147a838281c, 632b24d58d084af3a94c8270260024cd; no voters:
I20260806 16:03:05.195391 11996 raft_consensus.cc:2781] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:03:05.195495 11996 raft_consensus.cc:482] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:03:05.195557 11996 raft_consensus.cc:3037] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:05.196583 11996 raft_consensus.cc:504] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af0cb422e2f4ae99ff96147a838281c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37379 } } peers { permanent_uuid: "632b24d58d084af3a94c8270260024cd" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 36203 } } peers { permanent_uuid: "234b4e4d198c4a7a97900a4e542547ce" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43673 } }
I20260806 16:03:05.196918 11996 leader_election.cc:288] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [CANDIDATE]: Term 1 election: Requested vote from peers 632b24d58d084af3a94c8270260024cd (127.25.31.130:36203), 234b4e4d198c4a7a97900a4e542547ce (127.25.31.129:43673)
I20260806 16:03:05.197322 11792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a650e10c73cd4c54a5a2ab2aee511aa1" candidate_uuid: "4af0cb422e2f4ae99ff96147a838281c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "632b24d58d084af3a94c8270260024cd"
I20260806 16:03:05.197351 11658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a650e10c73cd4c54a5a2ab2aee511aa1" candidate_uuid: "4af0cb422e2f4ae99ff96147a838281c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "234b4e4d198c4a7a97900a4e542547ce"
I20260806 16:03:05.197463 11792 raft_consensus.cc:3037] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:05.197472 11658 raft_consensus.cc:3037] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:05.198549 11658 raft_consensus.cc:2454] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af0cb422e2f4ae99ff96147a838281c in term 1.
I20260806 16:03:05.198551 11792 raft_consensus.cc:2454] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4af0cb422e2f4ae99ff96147a838281c in term 1.
I20260806 16:03:05.198860 11859 leader_election.cc:302] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 234b4e4d198c4a7a97900a4e542547ce, 4af0cb422e2f4ae99ff96147a838281c; no voters:
I20260806 16:03:05.199074 11996 raft_consensus.cc:2781] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:05.199304 11996 raft_consensus.cc:686] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [term 1 LEADER]: Becoming Leader. State: Replica: 4af0cb422e2f4ae99ff96147a838281c, State: Running, Role: LEADER
I20260806 16:03:05.199633 11996 consensus_queue.cc:237] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af0cb422e2f4ae99ff96147a838281c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37379 } } peers { permanent_uuid: "632b24d58d084af3a94c8270260024cd" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 36203 } } peers { permanent_uuid: "234b4e4d198c4a7a97900a4e542547ce" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43673 } }
I20260806 16:03:05.202771 11511 catalog_manager.cc:5495] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4af0cb422e2f4ae99ff96147a838281c (127.25.31.131). New cstate: current_term: 1 leader_uuid: "4af0cb422e2f4ae99ff96147a838281c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4af0cb422e2f4ae99ff96147a838281c" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 37379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "632b24d58d084af3a94c8270260024cd" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 36203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "234b4e4d198c4a7a97900a4e542547ce" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43673 } health_report { overall_health: UNKNOWN } } }
W20260806 16:03:05.206267 11973 tablet.cc:2369] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:03:05.239183 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:05.235473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46122 (local address 127.25.31.131:37379)
0806 16:03:05.235601 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:05.235606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:05.235622 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:03:05.235807 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:05.235810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:05.235878 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:05.236008 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:05.236247 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.236757 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.237653 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:05.237874 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:05.237980 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:05.238292 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:05.238755 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:05.238887 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:05.238954 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 16:03:05.239015 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 16:03:05.244917 11658 raft_consensus.cc:1261] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce [term 1 FOLLOWER]: Refusing update from remote peer 4af0cb422e2f4ae99ff96147a838281c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:05.244968 11792 raft_consensus.cc:1261] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd [term 1 FOLLOWER]: Refusing update from remote peer 4af0cb422e2f4ae99ff96147a838281c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:05.245651 11996 consensus_queue.cc:1035] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [LEADER]: Connected to new peer: Peer: permanent_uuid: "234b4e4d198c4a7a97900a4e542547ce" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:03:05.245896 11999 consensus_queue.cc:1035] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [LEADER]: Connected to new peer: Peer: permanent_uuid: "632b24d58d084af3a94c8270260024cd" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 36203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:03:05.253183 12004 mvcc.cc:205] Tried to move back new op lower bound from 7315587830756012032 to 7315587830580371456. Current Snapshot: MvccSnapshot[applied={T|T < 7315587830756012032}]
I20260806 16:03:05.253647 12005 mvcc.cc:205] Tried to move back new op lower bound from 7315587830756012032 to 7315587830580371456. Current Snapshot: MvccSnapshot[applied={T|T < 7315587830756012032}]
I20260806 16:03:05.260692 11511 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:03:05.260862 11511 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:03:05.263376 11511 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6bfadf058f204b4ea2bfdae6a55241f4: Sending DeleteTablet for 3 replicas of tablet a650e10c73cd4c54a5a2ab2aee511aa1
I20260806 16:03:05.264104 11905 tablet_service.cc:1514] Processing DeleteTablet for tablet a650e10c73cd4c54a5a2ab2aee511aa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:05 UTC) from {username='slave'} at 127.0.0.1:46106
I20260806 16:03:05.264115 11771 tablet_service.cc:1514] Processing DeleteTablet for tablet a650e10c73cd4c54a5a2ab2aee511aa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:05 UTC) from {username='slave'} at 127.0.0.1:35676
I20260806 16:03:05.264577 12013 tablet_replica.cc:330] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c: stopping tablet replica
I20260806 16:03:05.264611 12014 tablet_replica.cc:330] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd: stopping tablet replica
I20260806 16:03:05.264580 11637 tablet_service.cc:1514] Processing DeleteTablet for tablet a650e10c73cd4c54a5a2ab2aee511aa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:05 UTC) from {username='slave'} at 127.0.0.1:44642
I20260806 16:03:05.264899 12014 raft_consensus.cc:2229] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:03:05.265199 12014 raft_consensus.cc:2258] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:03:05.265197 12013 raft_consensus.cc:2229] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:03:05.265499 12015 tablet_replica.cc:330] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce: stopping tablet replica
I20260806 16:03:05.265594 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 11574
I20260806 16:03:05.265774 12015 raft_consensus.cc:2229] T a650e10c73cd4c54a5a2ab2aee511aa1 P 234b4e4d198c4a7a97900a4e542547ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:03:05.265751 12013 raft_consensus.cc:2258] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:03:05.268553 12013 ts_tablet_manager.cc:1905] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:03:05.269942 12014 ts_tablet_manager.cc:1905] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:03:05.272065 12013 ts_tablet_manager.cc:1918] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:03:05.272179 12013 log.cc:1196] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/a650e10c73cd4c54a5a2ab2aee511aa1
I20260806 16:03:05.272545 12013 ts_tablet_manager.cc:1939] T a650e10c73cd4c54a5a2ab2aee511aa1 P 4af0cb422e2f4ae99ff96147a838281c: Deleting consensus metadata
I20260806 16:03:05.272629 12014 ts_tablet_manager.cc:1918] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:03:05.272730 12014 log.cc:1196] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/a650e10c73cd4c54a5a2ab2aee511aa1
I20260806 16:03:05.273042 12014 ts_tablet_manager.cc:1939] T a650e10c73cd4c54a5a2ab2aee511aa1 P 632b24d58d084af3a94c8270260024cd: Deleting consensus metadata
I20260806 16:03:05.273766 11499 catalog_manager.cc:4841] TS 4af0cb422e2f4ae99ff96147a838281c (127.25.31.131:37379): tablet a650e10c73cd4c54a5a2ab2aee511aa1 (table test-table [id=e3274d8c96cd43028c3877cf3beb7543]) successfully deleted
I20260806 16:03:05.274190 11498 catalog_manager.cc:4841] TS 632b24d58d084af3a94c8270260024cd (127.25.31.130:36203): tablet a650e10c73cd4c54a5a2ab2aee511aa1 (table test-table [id=e3274d8c96cd43028c3877cf3beb7543]) successfully deleted
W20260806 16:03:05.274811 11498 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:43673 (error 108)
I20260806 16:03:05.275005 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 11708
I20260806 16:03:05.279029 11987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:05.278656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:43673 (local address 127.0.0.1:44644)
0806 16:03:05.278835 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:05.278923 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:43673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 16:03:05.279222 11498 catalog_manager.cc:4584] TS 234b4e4d198c4a7a97900a4e542547ce (127.25.31.129:43673): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a650e10c73cd4c54a5a2ab2aee511aa1: Network error: Client connection negotiation failed: client connection to 127.25.31.129:43673: connect: Connection refused (error 111)
I20260806 16:03:05.282068 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 11841
I20260806 16:03:05.287739 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 11481
2026-08-06T16:03:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1178 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:05 dist-test-slave-tm1g krb5kdc[12019](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:05 dist-test-slave-tm1g krb5kdc[12019](info): set up 2 sockets
Aug 06 16:03:05 dist-test-slave-tm1g krb5kdc[12019](info): commencing operation
krb5kdc: starting...
W20260806 16:03:06.330933 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:06 dist-test-slave-tm1g krb5kdc[12019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:03:06.449054 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:33487
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:46405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:33487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:06.558544 12032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:06.558805 12032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:06.558876 12032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:06.562858 12032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:06.562929 12032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:06.562966 12032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:06.563024 12032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:06.563062 12032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:06.568162 12032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:33487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:33487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:06.569152 12032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:06.570132 12032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:06.578477 12037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:06.578504 12040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:06.578492 12038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:06.578483 12032 server_base.cc:1034] running on GCE node
I20260806 16:03:06.579262 12032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:06.580339 12032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:06.581542 12032 hybrid_clock.cc:648] HybridClock initialized: now 1786032186581528 us; error 35 us; skew 500 ppm
Aug 06 16:03:06 dist-test-slave-tm1g krb5kdc[12019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032186, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:06.584251 12032 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:03:06.585166 12032 webserver.cc:460] Webserver started at http://127.25.31.190:46223/ using document root <none> and password file <none>
I20260806 16:03:06.585700 12032 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:06.585760 12032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:06.585922 12032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:06.587944 12032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "01d8fb08f7e94c7684ed2886d84ed615"
format_stamp: "Formatted at 2026-08-06 16:03:06 on dist-test-slave-tm1g"
server_key: "7b2b9adf145ed06d75b334818b662106"
server_key_iv: "121332d33578cac05819f64c8c69d6d4"
server_key_version: "encryptionkey@0"
I20260806 16:03:06.588454 12032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "01d8fb08f7e94c7684ed2886d84ed615"
format_stamp: "Formatted at 2026-08-06 16:03:06 on dist-test-slave-tm1g"
server_key: "7b2b9adf145ed06d75b334818b662106"
server_key_iv: "121332d33578cac05819f64c8c69d6d4"
server_key_version: "encryptionkey@0"
I20260806 16:03:06.596371 12032 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.001s
I20260806 16:03:06.599264 12047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:06.600507 12032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:03:06.600667 12032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "01d8fb08f7e94c7684ed2886d84ed615"
format_stamp: "Formatted at 2026-08-06 16:03:06 on dist-test-slave-tm1g"
server_key: "7b2b9adf145ed06d75b334818b662106"
server_key_iv: "121332d33578cac05819f64c8c69d6d4"
server_key_version: "encryptionkey@0"
I20260806 16:03:06.600793 12032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:06.618746 12032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:06.621455 12032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:06.621655 12032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:06.628793 12032 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:33487
I20260806 16:03:06.628819 12100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:33487 every 8 connection(s)
I20260806 16:03:06.629716 12032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:06.632401 12101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:06.634907 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 12032
I20260806 16:03:06.635054 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:06.635365 25726 external_mini_cluster.cc:1428] Setting key 5101b0f53e74fa475f991eaba14c0b2c
I20260806 16:03:06.638633 12101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615: Bootstrap starting.
I20260806 16:03:06.640842 12101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:06.641583 12101 log.cc:824] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:06.643559 12101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615: No bootstrap required, opened a new log
Aug 06 16:03:06 dist-test-slave-tm1g krb5kdc[12019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:03:06.646996 12101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01d8fb08f7e94c7684ed2886d84ed615" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 33487 } }
I20260806 16:03:06.647217 12101 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:06.647285 12101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01d8fb08f7e94c7684ed2886d84ed615, State: Initialized, Role: FOLLOWER
I20260806 16:03:06.647707 12101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01d8fb08f7e94c7684ed2886d84ed615" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 33487 } }
I20260806 16:03:06.647861 12101 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:06.647949 12101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:06.648074 12101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:06.648937 12101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01d8fb08f7e94c7684ed2886d84ed615" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 33487 } }
I20260806 16:03:06.649284 12101 leader_election.cc:302] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01d8fb08f7e94c7684ed2886d84ed615; no voters:
I20260806 16:03:06.649644 12101 leader_election.cc:288] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:06.649731 12107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:06.649938 12107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [term 1 LEADER]: Becoming Leader. State: Replica: 01d8fb08f7e94c7684ed2886d84ed615, State: Running, Role: LEADER
I20260806 16:03:06.650246 12107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01d8fb08f7e94c7684ed2886d84ed615" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 33487 } }
I20260806 16:03:06.650672 12105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:06.636863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55022 (local address 127.25.31.190:33487)
0806 16:03:06.637301 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:06.637316 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:06.637354 (+ 38us) server_negotiation.cc:407] Connection header received
0806 16:03:06.637992 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:06.638014 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:06.638399 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:06.638721 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:06.639525 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:06.640850 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:06.641710 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:06.642186 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:06.642386 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:06.645060 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:06.645066 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:06.645078 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:06.645103 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:06.647587 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:06.648062 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:06.648068 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:06.648073 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:06.648224 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:06.648498 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:06.648501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:06.648503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:06.648895 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:06.649114 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:06.649351 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 16:03:06.649674 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":139,"threads_started":1}
I20260806 16:03:06.650974 12101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:06.651440 12107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01d8fb08f7e94c7684ed2886d84ed615. Latest consensus state: current_term: 1 leader_uuid: "01d8fb08f7e94c7684ed2886d84ed615" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01d8fb08f7e94c7684ed2886d84ed615" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 33487 } } }
I20260806 16:03:06.651551 12107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:06.652170 12108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01d8fb08f7e94c7684ed2886d84ed615" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01d8fb08f7e94c7684ed2886d84ed615" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 33487 } } }
I20260806 16:03:06.652275 12108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:06.653280 12111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:06.656375 12111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:06.662659 12111 catalog_manager.cc:1347] Generated new cluster ID: 24c6a2fc05354c84ba361579eccde407
I20260806 16:03:06.662737 12111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:06.685181 12111 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:06.686517 12111 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:06.696768 12111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615: Generated new TSK 0
I20260806 16:03:06.697594 12111 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 16:03:06.737064 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:33487
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:46405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:06.847628 12128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:06.847882 12128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:06.847925 12128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:06.851965 12128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:06.852027 12128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:06.852114 12128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:03:06.857451 12128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:33487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:06.858579 12128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:06.859591 12128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:06.869059 12133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:06.869047 12136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:06.869122 12134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:06.869264 12128 server_base.cc:1034] running on GCE node
I20260806 16:03:06.869693 12128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:06.870357 12128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:06.871537 12128 hybrid_clock.cc:648] HybridClock initialized: now 1786032186871513 us; error 35 us; skew 500 ppm
Aug 06 16:03:06 dist-test-slave-tm1g krb5kdc[12019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032186, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:06.874212 12128 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 16:03:06.875104 12128 webserver.cc:460] Webserver started at http://127.25.31.129:35793/ using document root <none> and password file <none>
I20260806 16:03:06.875653 12128 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:06.875715 12128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:06.875946 12128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:06.877946 12128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "7e925333f94445d397dee2195a238231"
format_stamp: "Formatted at 2026-08-06 16:03:06 on dist-test-slave-tm1g"
server_key: "c011ab6058db2d2f64e6c8b56fdab4e2"
server_key_iv: "b30243a02fd2b7323dde91ae4188aba1"
server_key_version: "encryptionkey@0"
I20260806 16:03:06.878535 12128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "7e925333f94445d397dee2195a238231"
format_stamp: "Formatted at 2026-08-06 16:03:06 on dist-test-slave-tm1g"
server_key: "c011ab6058db2d2f64e6c8b56fdab4e2"
server_key_iv: "b30243a02fd2b7323dde91ae4188aba1"
server_key_version: "encryptionkey@0"
I20260806 16:03:06.882091 12128 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:03:06.884500 12143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:06.885725 12128 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:03:06.885820 12128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "7e925333f94445d397dee2195a238231"
format_stamp: "Formatted at 2026-08-06 16:03:06 on dist-test-slave-tm1g"
server_key: "c011ab6058db2d2f64e6c8b56fdab4e2"
server_key_iv: "b30243a02fd2b7323dde91ae4188aba1"
server_key_version: "encryptionkey@0"
I20260806 16:03:06.885948 12128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:06.902971 12128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:06.906071 12128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:06.906220 12128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:06.906810 12128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:06.907580 12128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:06.907627 12128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:06.907666 12128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:06.907682 12128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:06.916326 12128 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:37757
I20260806 16:03:06.916438 12258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:37757 every 8 connection(s)
I20260806 16:03:06.917223 12128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 16:03:06.923411 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 12128
I20260806 16:03:06.923552 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 16:03:06.923839 25726 external_mini_cluster.cc:1428] Setting key ea3b814a72f107054ecce29f45f09ec8
Aug 06 16:03:06 dist-test-slave-tm1g krb5kdc[12019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032186, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:03:06.931638 12105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:06.919011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:56239 (local address 127.25.31.190:33487)
0806 16:03:06.919155 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:06.919160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:06.919998 (+ 838us) server_negotiation.cc:407] Connection header received
0806 16:03:06.921255 (+ 1257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:06.921258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:06.921305 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:06.921382 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:06.922537 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:06.923079 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:06.924440 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:06.924714 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:06.924797 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:06.927595 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:06.927599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:06.927601 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:06.927622 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:06.929327 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:06.929906 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:06.929912 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:06.929913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:06.930010 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:06.930356 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:06.930362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:06.930375 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:06.930640 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:06.930754 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:06.931232 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 16:03:06.931390 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 16:03:06.932196 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:06.919424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:33487 (local address 127.25.31.129:56239)
0806 16:03:06.919846 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:06.919887 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:06.921062 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:06.921538 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:06.921544 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:06.921864 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:06.922405 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:06.922421 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:06.923197 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:06.923199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:06.924314 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:06.924329 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:06.924867 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:06.924870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:06.925079 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:06.925757 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:06.925773 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:06.927432 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:06.929444 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:06.929448 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:06.929458 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:06.929791 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:06.930109 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:06.930112 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:06.930113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:06.930240 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:06.930760 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:06.930764 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:06.930982 (+ 218us) client_negotiation.cc:769] Sending connection context
0806 16:03:06.931143 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 16:03:06.931377 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":115,"threads_started":1}
I20260806 16:03:06.933429 12259 heartbeater.cc:347] Connected to a master server at 127.25.31.190:33487
I20260806 16:03:06.933650 12259 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
I20260806 16:03:06.934231 12259 heartbeater.cc:511] Master 127.25.31.190:33487 requested a full tablet report, sending...
I20260806 16:03:06.936067 12064 ts_manager.cc:194] Registered new tserver with Master: 7e925333f94445d397dee2195a238231 (127.25.31.129:37757)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
I20260806 16:03:06.937760 12064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:56239
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 16:03:06.953147 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:33487
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:46405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:07.063040 12264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:07.063331 12264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:07.063422 12264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:07.067408 12264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:07.067492 12264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:07.067587 12264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:03:07.072703 12264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:33487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:07.073745 12264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:07.074787 12264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:07.084088 12270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:07.084088 12272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:07.084102 12269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:07.084482 12264 server_base.cc:1034] running on GCE node
I20260806 16:03:07.084901 12264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:07.085520 12264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:07.086674 12264 hybrid_clock.cc:648] HybridClock initialized: now 1786032187086638 us; error 47 us; skew 500 ppm
Aug 06 16:03:07 dist-test-slave-tm1g krb5kdc[12019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032187, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:07.089332 12264 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 16:03:07.090219 12264 webserver.cc:460] Webserver started at http://127.25.31.130:43501/ using document root <none> and password file <none>
I20260806 16:03:07.090790 12264 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:07.090900 12264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:07.091128 12264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:07.093173 12264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0"
format_stamp: "Formatted at 2026-08-06 16:03:07 on dist-test-slave-tm1g"
server_key: "15057545c948fa926236720626c37543"
server_key_iv: "38f69d5ac8084c7e9db72ce071d615b0"
server_key_version: "encryptionkey@0"
I20260806 16:03:07.093734 12264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0"
format_stamp: "Formatted at 2026-08-06 16:03:07 on dist-test-slave-tm1g"
server_key: "15057545c948fa926236720626c37543"
server_key_iv: "38f69d5ac8084c7e9db72ce071d615b0"
server_key_version: "encryptionkey@0"
I20260806 16:03:07.097426 12264 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 16:03:07.099869 12279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:07.101178 12264 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:03:07.101336 12264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0"
format_stamp: "Formatted at 2026-08-06 16:03:07 on dist-test-slave-tm1g"
server_key: "15057545c948fa926236720626c37543"
server_key_iv: "38f69d5ac8084c7e9db72ce071d615b0"
server_key_version: "encryptionkey@0"
I20260806 16:03:07.101456 12264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:07.112377 12264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:07.115285 12264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:07.115494 12264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:07.116048 12264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:07.116849 12264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:07.116920 12264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:07.116993 12264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:07.117053 12264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:07.126163 12264 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:34367
I20260806 16:03:07.126192 12392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:34367 every 8 connection(s)
I20260806 16:03:07.127069 12264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:03:07.129130 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 12264
I20260806 16:03:07.129264 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 16:03:07.129593 25726 external_mini_cluster.cc:1428] Setting key 3f2f5f6fe362d0b8481c582c0ce95f69
Aug 06 16:03:07 dist-test-slave-tm1g krb5kdc[12019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032187, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
I20260806 16:03:07.143080 12105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.128859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:49725 (local address 127.25.31.190:33487)
0806 16:03:07.129022 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:07.129028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:07.129943 (+ 915us) server_negotiation.cc:407] Connection header received
0806 16:03:07.131626 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:07.131628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:07.131673 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:07.131737 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:07.133369 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.134261 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.136076 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.136298 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.136382 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.139328 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:07.139337 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:07.139339 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:07.139362 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:07.140947 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:07.141472 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:07.141476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:07.141480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:07.141559 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:07.141919 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:07.141922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:07.141924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:07.142107 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:07.142217 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:07.142781 (+ 564us) server_negotiation.cc:299] Negotiation successful
0806 16:03:07.142906 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
I20260806 16:03:07.143936 12396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.129219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:33487 (local address 127.25.31.130:49725)
0806 16:03:07.129772 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:07.129825 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:07.131381 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:07.131934 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:07.131945 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:07.132496 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:07.133189 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.133214 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.134420 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.134423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.135928 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.135944 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.136459 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.136467 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.136701 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.137324 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:07.137351 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:07.139154 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:07.141078 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:07.141084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:07.141098 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:07.141357 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:07.141655 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:07.141659 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:07.141661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:07.141801 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:07.142214 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:07.142220 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:07.142536 (+ 316us) client_negotiation.cc:769] Sending connection context
0806 16:03:07.142723 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 16:03:07.143049 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":149,"threads_started":1}
I20260806 16:03:07.145185 12393 heartbeater.cc:347] Connected to a master server at 127.25.31.190:33487
I20260806 16:03:07.145459 12393 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:07.145952 12393 heartbeater.cc:511] Master 127.25.31.190:33487 requested a full tablet report, sending...
I20260806 16:03:07.147096 12064 ts_manager.cc:194] Registered new tserver with Master: 0e71cbd27c5f43d9acb36cdfa007d6b0 (127.25.31.130:34367)
I20260806 16:03:07.147964 12064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:49725
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 16:03:07.159329 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:33487
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:46405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:07.270223 12399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:07.270535 12399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:07.270601 12399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:07.274605 12399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:07.274657 12399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:07.274712 12399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:03:07.279883 12399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:33487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:07.281000 12399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:07.281989 12399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:07.291945 12405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:07.291962 12404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:07.291998 12407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:07.292376 12399 server_base.cc:1034] running on GCE node
I20260806 16:03:07.292721 12399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:07.293295 12399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:07.294438 12399 hybrid_clock.cc:648] HybridClock initialized: now 1786032187294431 us; error 39 us; skew 500 ppm
Aug 06 16:03:07 dist-test-slave-tm1g krb5kdc[12019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032187, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:07.297147 12399 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 16:03:07.297994 12399 webserver.cc:460] Webserver started at http://127.25.31.131:35715/ using document root <none> and password file <none>
I20260806 16:03:07.298610 12399 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:07.298704 12399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:07.298934 12399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:07.300920 12399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "d861c972a8d3468789d31d24839c26c3"
format_stamp: "Formatted at 2026-08-06 16:03:07 on dist-test-slave-tm1g"
server_key: "0f14c794e5b32383b95cacc664f94d52"
server_key_iv: "987aa2957d1ffcdba76851bb6627b546"
server_key_version: "encryptionkey@0"
I20260806 16:03:07.301501 12399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "d861c972a8d3468789d31d24839c26c3"
format_stamp: "Formatted at 2026-08-06 16:03:07 on dist-test-slave-tm1g"
server_key: "0f14c794e5b32383b95cacc664f94d52"
server_key_iv: "987aa2957d1ffcdba76851bb6627b546"
server_key_version: "encryptionkey@0"
I20260806 16:03:07.305343 12399 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 16:03:07.307806 12414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:07.308929 12399 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:03:07.309072 12399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "d861c972a8d3468789d31d24839c26c3"
format_stamp: "Formatted at 2026-08-06 16:03:07 on dist-test-slave-tm1g"
server_key: "0f14c794e5b32383b95cacc664f94d52"
server_key_iv: "987aa2957d1ffcdba76851bb6627b546"
server_key_version: "encryptionkey@0"
I20260806 16:03:07.309181 12399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:07.329790 12399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:07.333180 12399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:07.333393 12399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:07.334007 12399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:07.334883 12399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:07.334959 12399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:07.335044 12399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:07.335099 12399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:07.343833 12399 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:35565
I20260806 16:03:07.343865 12528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:35565 every 8 connection(s)
I20260806 16:03:07.344725 12399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:03:07.345489 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 12399
I20260806 16:03:07.345638 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 16:03:07.345949 25726 external_mini_cluster.cc:1428] Setting key 253eedbecf9909a9937686ec4ed36778
Aug 06 16:03:07 dist-test-slave-tm1g krb5kdc[12019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032187, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:03:07.358333 12105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.347974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:40131 (local address 127.25.31.190:33487)
0806 16:03:07.348104 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:07.348109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:07.348124 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:03:07.349171 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:07.349173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:07.349219 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:07.349288 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:07.350440 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.350975 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.352154 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.352418 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.352499 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.355053 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:07.355057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:07.355059 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:07.355082 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:07.356293 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:07.356807 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:07.356812 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:07.356813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:07.356898 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:07.357249 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:07.357253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:07.357255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:07.357437 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:07.357556 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:07.358017 (+ 461us) server_negotiation.cc:299] Negotiation successful
0806 16:03:07.358142 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 16:03:07.359323 12531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.347014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:33487 (local address 127.25.31.131:40131)
0806 16:03:07.347597 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:07.347656 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:07.348982 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:07.349452 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:07.349458 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:07.349797 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:07.350259 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.350272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.351080 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.351083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.352050 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.352059 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.352553 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.352556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.352699 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.353296 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:07.353313 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:07.354876 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:07.356423 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:07.356429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:07.356443 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:07.356700 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:07.356995 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:07.356999 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:07.357001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:07.357149 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:07.357548 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:07.357554 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:07.357789 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 16:03:07.357972 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 16:03:07.358295 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":150,"threads_started":1}
I20260806 16:03:07.360380 12529 heartbeater.cc:347] Connected to a master server at 127.25.31.190:33487
I20260806 16:03:07.360597 12529 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:07.361017 12529 heartbeater.cc:511] Master 127.25.31.190:33487 requested a full tablet report, sending...
I20260806 16:03:07.361967 12063 ts_manager.cc:194] Registered new tserver with Master: d861c972a8d3468789d31d24839c26c3 (127.25.31.131:35565)
I20260806 16:03:07.362614 12063 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:40131
I20260806 16:03:07.371884 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:03:07.381785 12105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.373902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55026 (local address 127.25.31.190:33487)
0806 16:03:07.374053 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:07.374057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:07.374224 (+ 167us) server_negotiation.cc:407] Connection header received
0806 16:03:07.374343 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:07.374346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:07.374398 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:07.374470 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:07.375148 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.375679 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.376570 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.376788 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.376921 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.378644 (+ 1723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:07.378649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:07.378651 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:07.378696 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:07.379913 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:07.380458 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:07.380462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:07.380463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:07.380545 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:07.380886 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:07.380889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:07.380891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:07.381090 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:07.381249 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:07.381538 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 16:03:07.381632 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 16:03:07.384599 12063 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:03:07.388034 12063 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:03:07.402462 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.397090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57326 (local address 127.25.31.129:37757)
0806 16:03:07.397893 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:07.397908 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:07.397929 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:03:07.398004 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:07.398009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:07.398207 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:07.398666 (+ 459us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:07.398825 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.399793 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.401107 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.401786 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.401917 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.402060 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:07.402207 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 16:03:07.402322 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":123,"threads_started":1}
I20260806 16:03:07.402457 12542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.396751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:37757 (local address 127.0.0.1:57326)
0806 16:03:07.397380 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:07.397401 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:07.397513 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:07.398388 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:07.398391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:07.398408 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:07.398511 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.398518 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.399937 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.399940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.400989 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.400998 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.401910 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.401914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.402023 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.402058 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 16:03:07.402168 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 16:03:07.402284 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":68,"thread_start_us":117,"threads_started":1}
I20260806 16:03:07.403249 12545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.397155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:35565 (local address 127.0.0.1:37518)
0806 16:03:07.398278 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:07.398290 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:07.398407 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:07.398910 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:07.398913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:07.398931 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:07.399025 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.399031 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.400174 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.400177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.401294 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.401302 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.402870 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.402876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.402957 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.402981 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:03:07.403049 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 16:03:07.403119 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1066,"mutex_wait_us":2,"thread_start_us":103,"threads_started":1}
I20260806 16:03:07.403326 12543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.396738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47226 (local address 127.25.31.130:34367)
0806 16:03:07.397196 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:07.397210 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:07.397238 (+ 28us) server_negotiation.cc:407] Connection header received
0806 16:03:07.397534 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:07.397544 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:07.397761 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:07.397921 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:07.398175 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.399364 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.401365 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.402220 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.402456 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.402593 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:07.403090 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 16:03:07.403187 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":143,"threads_started":1}
I20260806 16:03:07.403715 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.396476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:34367 (local address 127.0.0.1:47226)
0806 16:03:07.397107 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:07.397156 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:07.397282 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:07.397919 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:07.397925 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:07.397953 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:07.398051 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.398056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.399519 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.399522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.401233 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.401241 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.402697 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.402701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.402788 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.402807 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 16:03:07.403507 (+ 700us) client_negotiation.cc:240] Negotiation successful
0806 16:03:07.403585 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":333,"thread_start_us":68,"threads_started":1}
I20260806 16:03:07.404047 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.397581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37518 (local address 127.25.31.131:35565)
0806 16:03:07.397872 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:07.397884 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:07.398521 (+ 637us) server_negotiation.cc:407] Connection header received
0806 16:03:07.398605 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:07.398610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:07.398771 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:07.398922 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:07.399149 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.400034 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.401420 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.402221 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.402351 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.402501 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:07.403694 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0806 16:03:07.403897 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":108,"threads_started":1}
I20260806 16:03:07.405609 12192 tablet_service.cc:1467] Processing CreateTablet for tablet acdb8c6762114ea29c074e7ae5476f9d (DEFAULT_TABLE table=test-table [id=c2b6fd53beaf47618f35e5b847dfcd60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:07.406483 12327 tablet_service.cc:1467] Processing CreateTablet for tablet acdb8c6762114ea29c074e7ae5476f9d (DEFAULT_TABLE table=test-table [id=c2b6fd53beaf47618f35e5b847dfcd60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:07.406642 12462 tablet_service.cc:1467] Processing CreateTablet for tablet acdb8c6762114ea29c074e7ae5476f9d (DEFAULT_TABLE table=test-table [id=c2b6fd53beaf47618f35e5b847dfcd60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:07.407027 12192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acdb8c6762114ea29c074e7ae5476f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:07.407436 12327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acdb8c6762114ea29c074e7ae5476f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:07.407475 12462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acdb8c6762114ea29c074e7ae5476f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:07.413618 12547 tablet_bootstrap.cc:492] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3: Bootstrap starting.
I20260806 16:03:07.414736 12549 tablet_bootstrap.cc:492] T acdb8c6762114ea29c074e7ae5476f9d P 7e925333f94445d397dee2195a238231: Bootstrap starting.
I20260806 16:03:07.415238 12548 tablet_bootstrap.cc:492] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0: Bootstrap starting.
I20260806 16:03:07.416508 12549 tablet_bootstrap.cc:654] T acdb8c6762114ea29c074e7ae5476f9d P 7e925333f94445d397dee2195a238231: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:07.416514 12547 tablet_bootstrap.cc:654] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:07.417266 12548 tablet_bootstrap.cc:654] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:07.417640 12549 log.cc:824] T acdb8c6762114ea29c074e7ae5476f9d P 7e925333f94445d397dee2195a238231: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:07.417882 12547 log.cc:824] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:07.418555 12548 log.cc:824] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:07.419860 12549 tablet_bootstrap.cc:492] T acdb8c6762114ea29c074e7ae5476f9d P 7e925333f94445d397dee2195a238231: No bootstrap required, opened a new log
I20260806 16:03:07.420099 12549 ts_tablet_manager.cc:1397] T acdb8c6762114ea29c074e7ae5476f9d P 7e925333f94445d397dee2195a238231: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 16:03:07.420142 12547 tablet_bootstrap.cc:492] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3: No bootstrap required, opened a new log
I20260806 16:03:07.420220 12548 tablet_bootstrap.cc:492] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0: No bootstrap required, opened a new log
I20260806 16:03:07.420296 12547 ts_tablet_manager.cc:1397] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 16:03:07.420401 12548 ts_tablet_manager.cc:1397] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 16:03:07.423323 12548 raft_consensus.cc:348] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34367 } } peers { permanent_uuid: "7e925333f94445d397dee2195a238231" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37757 } } peers { permanent_uuid: "d861c972a8d3468789d31d24839c26c3" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35565 } }
I20260806 16:03:07.423574 12548 raft_consensus.cc:374] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:07.423630 12548 raft_consensus.cc:729] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e71cbd27c5f43d9acb36cdfa007d6b0, State: Initialized, Role: FOLLOWER
I20260806 16:03:07.424068 12548 consensus_queue.cc:260] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34367 } } peers { permanent_uuid: "7e925333f94445d397dee2195a238231" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37757 } } peers { permanent_uuid: "d861c972a8d3468789d31d24839c26c3" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35565 } }
I20260806 16:03:07.424010 12547 raft_consensus.cc:348] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34367 } } peers { permanent_uuid: "7e925333f94445d397dee2195a238231" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37757 } } peers { permanent_uuid: "d861c972a8d3468789d31d24839c26c3" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35565 } }
I20260806 16:03:07.424019 12549 raft_consensus.cc:348] T acdb8c6762114ea29c074e7ae5476f9d P 7e925333f94445d397dee2195a238231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34367 } } peers { permanent_uuid: "7e925333f94445d397dee2195a238231" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37757 } } peers { permanent_uuid: "d861c972a8d3468789d31d24839c26c3" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35565 } }
I20260806 16:03:07.424257 12549 raft_consensus.cc:374] T acdb8c6762114ea29c074e7ae5476f9d P 7e925333f94445d397dee2195a238231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:07.424257 12547 raft_consensus.cc:374] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:07.424317 12547 raft_consensus.cc:729] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d861c972a8d3468789d31d24839c26c3, State: Initialized, Role: FOLLOWER
I20260806 16:03:07.424326 12549 raft_consensus.cc:729] T acdb8c6762114ea29c074e7ae5476f9d P 7e925333f94445d397dee2195a238231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e925333f94445d397dee2195a238231, State: Initialized, Role: FOLLOWER
I20260806 16:03:07.424751 12547 consensus_queue.cc:260] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34367 } } peers { permanent_uuid: "7e925333f94445d397dee2195a238231" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37757 } } peers { permanent_uuid: "d861c972a8d3468789d31d24839c26c3" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35565 } }
I20260806 16:03:07.424751 12549 consensus_queue.cc:260] T acdb8c6762114ea29c074e7ae5476f9d P 7e925333f94445d397dee2195a238231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34367 } } peers { permanent_uuid: "7e925333f94445d397dee2195a238231" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37757 } } peers { permanent_uuid: "d861c972a8d3468789d31d24839c26c3" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35565 } }
I20260806 16:03:07.425026 12548 ts_tablet_manager.cc:1428] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 16:03:07.425400 12393 heartbeater.cc:503] Master 127.25.31.190:33487 was elected leader, sending a full tablet report...
I20260806 16:03:07.425698 12549 ts_tablet_manager.cc:1428] T acdb8c6762114ea29c074e7ae5476f9d P 7e925333f94445d397dee2195a238231: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 16:03:07.425698 12547 ts_tablet_manager.cc:1428] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 16:03:07.426028 12259 heartbeater.cc:503] Master 127.25.31.190:33487 was elected leader, sending a full tablet report...
I20260806 16:03:07.426399 12529 heartbeater.cc:503] Master 127.25.31.190:33487 was elected leader, sending a full tablet report...
I20260806 16:03:07.453393 12553 raft_consensus.cc:482] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:03:07.453645 12553 raft_consensus.cc:504] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34367 } } peers { permanent_uuid: "7e925333f94445d397dee2195a238231" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37757 } } peers { permanent_uuid: "d861c972a8d3468789d31d24839c26c3" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35565 } }
I20260806 16:03:07.454746 12553 leader_election.cc:288] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e925333f94445d397dee2195a238231 (127.25.31.129:37757), d861c972a8d3468789d31d24839c26c3 (127.25.31.131:35565)
I20260806 16:03:07.458449 12396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.455042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:37757 (local address 127.25.31.130:57509)
0806 16:03:07.455188 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:07.455206 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:07.455303 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:07.455654 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:07.455657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:07.455675 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:07.455788 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.455793 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.456553 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.456556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.457355 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.457364 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.457995 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.457999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.458106 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.458133 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 16:03:07.458222 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 16:03:07.458298 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 16:03:07.458448 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.455128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:57509 (local address 127.25.31.129:37757)
0806 16:03:07.455275 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:07.455279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:07.455294 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:03:07.455441 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:07.455446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:07.455513 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:07.455595 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:07.455915 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.456435 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.457481 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.457875 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.458003 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.458055 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:07.458239 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 16:03:07.458343 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:03:07.458945 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.455321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:35565 (local address 127.25.31.130:53495)
0806 16:03:07.455661 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:07.455678 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:07.455786 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:07.456023 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:07.456026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:07.456043 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:07.456179 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.456184 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.456884 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.456889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.457663 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.457672 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.458507 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.458511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.458632 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.458669 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 16:03:07.458743 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 16:03:07.458812 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":150,"threads_started":1}
I20260806 16:03:07.458948 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.455451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:53495 (local address 127.25.31.131:35565)
0806 16:03:07.455585 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:07.455590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:07.455751 (+ 161us) server_negotiation.cc:407] Connection header received
0806 16:03:07.455875 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:07.455883 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:07.455924 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:07.456002 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:07.456287 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.456771 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.457788 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.458196 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.458344 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.458391 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:07.458761 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 16:03:07.458844 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 16:03:07.459095 12212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdb8c6762114ea29c074e7ae5476f9d" candidate_uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e925333f94445d397dee2195a238231" is_pre_election: true
I20260806 16:03:07.459476 12212 raft_consensus.cc:2454] T acdb8c6762114ea29c074e7ae5476f9d P 7e925333f94445d397dee2195a238231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e71cbd27c5f43d9acb36cdfa007d6b0 in term 0.
I20260806 16:03:07.459846 12482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdb8c6762114ea29c074e7ae5476f9d" candidate_uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d861c972a8d3468789d31d24839c26c3" is_pre_election: true
I20260806 16:03:07.460021 12282 leader_election.cc:302] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e71cbd27c5f43d9acb36cdfa007d6b0, 7e925333f94445d397dee2195a238231; no voters:
I20260806 16:03:07.460129 12482 raft_consensus.cc:2454] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e71cbd27c5f43d9acb36cdfa007d6b0 in term 0.
I20260806 16:03:07.460271 12553 raft_consensus.cc:2781] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:03:07.460355 12553 raft_consensus.cc:482] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:03:07.460428 12553 raft_consensus.cc:3037] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:07.461498 12553 raft_consensus.cc:504] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34367 } } peers { permanent_uuid: "7e925333f94445d397dee2195a238231" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37757 } } peers { permanent_uuid: "d861c972a8d3468789d31d24839c26c3" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35565 } }
I20260806 16:03:07.461863 12553 leader_election.cc:288] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [CANDIDATE]: Term 1 election: Requested vote from peers 7e925333f94445d397dee2195a238231 (127.25.31.129:37757), d861c972a8d3468789d31d24839c26c3 (127.25.31.131:35565)
I20260806 16:03:07.462246 12212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdb8c6762114ea29c074e7ae5476f9d" candidate_uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e925333f94445d397dee2195a238231"
I20260806 16:03:07.462332 12482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acdb8c6762114ea29c074e7ae5476f9d" candidate_uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d861c972a8d3468789d31d24839c26c3"
I20260806 16:03:07.462407 12212 raft_consensus.cc:3037] T acdb8c6762114ea29c074e7ae5476f9d P 7e925333f94445d397dee2195a238231 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:07.462442 12482 raft_consensus.cc:3037] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:07.463704 12212 raft_consensus.cc:2454] T acdb8c6762114ea29c074e7ae5476f9d P 7e925333f94445d397dee2195a238231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e71cbd27c5f43d9acb36cdfa007d6b0 in term 1.
I20260806 16:03:07.463704 12482 raft_consensus.cc:2454] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e71cbd27c5f43d9acb36cdfa007d6b0 in term 1.
I20260806 16:03:07.464007 12282 leader_election.cc:302] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e71cbd27c5f43d9acb36cdfa007d6b0, d861c972a8d3468789d31d24839c26c3; no voters:
I20260806 16:03:07.464205 12553 raft_consensus.cc:2781] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:07.464439 12553 raft_consensus.cc:686] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [term 1 LEADER]: Becoming Leader. State: Replica: 0e71cbd27c5f43d9acb36cdfa007d6b0, State: Running, Role: LEADER
I20260806 16:03:07.464753 12553 consensus_queue.cc:237] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34367 } } peers { permanent_uuid: "7e925333f94445d397dee2195a238231" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37757 } } peers { permanent_uuid: "d861c972a8d3468789d31d24839c26c3" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35565 } }
I20260806 16:03:07.468063 12063 catalog_manager.cc:5495] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e71cbd27c5f43d9acb36cdfa007d6b0 (127.25.31.130). New cstate: current_term: 1 leader_uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e71cbd27c5f43d9acb36cdfa007d6b0" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 34367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e925333f94445d397dee2195a238231" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d861c972a8d3468789d31d24839c26c3" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35565 } health_report { overall_health: UNKNOWN } } }
I20260806 16:03:07.483371 12543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.479892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47230 (local address 127.25.31.130:34367)
0806 16:03:07.480030 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:07.480033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:07.480047 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:03:07.480153 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:07.480155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:07.480204 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:07.480292 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:07.480506 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.481120 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.481978 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.482190 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.482265 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.482621 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:07.483004 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:07.483114 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:07.483214 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 16:03:07.483247 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 16:03:07.488914 12212 raft_consensus.cc:1261] T acdb8c6762114ea29c074e7ae5476f9d P 7e925333f94445d397dee2195a238231 [term 1 FOLLOWER]: Refusing update from remote peer 0e71cbd27c5f43d9acb36cdfa007d6b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:07.488917 12482 raft_consensus.cc:1261] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3 [term 1 FOLLOWER]: Refusing update from remote peer 0e71cbd27c5f43d9acb36cdfa007d6b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:07.489615 12557 consensus_queue.cc:1035] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e925333f94445d397dee2195a238231" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:03:07.489859 12553 consensus_queue.cc:1035] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d861c972a8d3468789d31d24839c26c3" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:03:07.496228 12560 mvcc.cc:205] Tried to move back new op lower bound from 7315587839947550720 to 7315587839857983488. Current Snapshot: MvccSnapshot[applied={T|T < 7315587839947550720}]
I20260806 16:03:07.498337 12561 mvcc.cc:205] Tried to move back new op lower bound from 7315587839947550720 to 7315587839857983488. Current Snapshot: MvccSnapshot[applied={T|T < 7315587839947550720}]
I20260806 16:03:07.498521 12562 mvcc.cc:205] Tried to move back new op lower bound from 7315587839947550720 to 7315587839857983488. Current Snapshot: MvccSnapshot[applied={T|T < 7315587839947550720}]
I20260806 16:03:07.505262 12063 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:03:07.505462 12063 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:03:07.507848 12063 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01d8fb08f7e94c7684ed2886d84ed615: Sending DeleteTablet for 3 replicas of tablet acdb8c6762114ea29c074e7ae5476f9d
I20260806 16:03:07.508555 12327 tablet_service.cc:1514] Processing DeleteTablet for tablet acdb8c6762114ea29c074e7ae5476f9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:07 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:47226
I20260806 16:03:07.508651 12462 tablet_service.cc:1514] Processing DeleteTablet for tablet acdb8c6762114ea29c074e7ae5476f9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:07 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:37518
I20260806 16:03:07.508785 12192 tablet_service.cc:1514] Processing DeleteTablet for tablet acdb8c6762114ea29c074e7ae5476f9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:07 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:57326
I20260806 16:03:07.509069 12569 tablet_replica.cc:330] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0: stopping tablet replica
I20260806 16:03:07.509424 12569 raft_consensus.cc:2229] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:03:07.509562 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 12128
I20260806 16:03:07.509835 12569 raft_consensus.cc:2258] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:03:07.511667 12569 ts_tablet_manager.cc:1905] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:03:07.514562 12569 ts_tablet_manager.cc:1918] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:03:07.514662 12569 log.cc:1196] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/acdb8c6762114ea29c074e7ae5476f9d
I20260806 16:03:07.515003 12569 ts_tablet_manager.cc:1939] T acdb8c6762114ea29c074e7ae5476f9d P 0e71cbd27c5f43d9acb36cdfa007d6b0: Deleting consensus metadata
I20260806 16:03:07.515640 12570 tablet_replica.cc:330] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3: stopping tablet replica
I20260806 16:03:07.516023 12570 raft_consensus.cc:2229] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:03:07.516235 12049 catalog_manager.cc:4841] TS 0e71cbd27c5f43d9acb36cdfa007d6b0 (127.25.31.130:34367): tablet acdb8c6762114ea29c074e7ae5476f9d (table test-table [id=c2b6fd53beaf47618f35e5b847dfcd60]) successfully deleted
I20260806 16:03:07.516336 12570 raft_consensus.cc:2258] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:03:07.517683 12570 ts_tablet_manager.cc:1905] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:03:07.520282 12570 ts_tablet_manager.cc:1918] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:03:07.520380 12570 log.cc:1196] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/acdb8c6762114ea29c074e7ae5476f9d
I20260806 16:03:07.520725 12570 ts_tablet_manager.cc:1939] T acdb8c6762114ea29c074e7ae5476f9d P d861c972a8d3468789d31d24839c26c3: Deleting consensus metadata
W20260806 16:03:07.521490 12050 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:37757 (error 108)
I20260806 16:03:07.521997 12050 catalog_manager.cc:4841] TS d861c972a8d3468789d31d24839c26c3 (127.25.31.131:35565): tablet acdb8c6762114ea29c074e7ae5476f9d (table test-table [id=c2b6fd53beaf47618f35e5b847dfcd60]) successfully deleted
I20260806 16:03:07.522505 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 12264
I20260806 16:03:07.522598 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.522269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:37757 (local address 127.0.0.1:57342)
0806 16:03:07.522428 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:07.522511 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:37757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 16:03:07.525925 12050 catalog_manager.cc:4584] TS 7e925333f94445d397dee2195a238231 (127.25.31.129:37757): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet acdb8c6762114ea29c074e7ae5476f9d: Network error: Client connection negotiation failed: client connection to 127.25.31.129:37757: connect: Connection refused (error 111)
I20260806 16:03:07.530902 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 12399
I20260806 16:03:07.537133 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 12032
2026-08-06T16:03:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2250 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T16:03:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:07Z Disabled control of system clock
I20260806 16:03:07.568091 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42747
--webserver_interface=127.25.31.190
--webserver_port=0
--builtin_ntp_servers=127.25.31.148:46111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:42747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:03:07.677456 12576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:07.677708 12576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:07.677743 12576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:07.681725 12576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:07.681792 12576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:07.681807 12576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:07.681826 12576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:07.681841 12576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:07.686733 12576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:42747
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:42747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:07.687762 12576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:07.689116 12576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:07.697613 12581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:07.697582 12582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:07.697754 12576 server_base.cc:1034] running on GCE node
W20260806 16:03:07.697600 12584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:07.698506 12576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:07.699545 12576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:07.700711 12576 hybrid_clock.cc:648] HybridClock initialized: now 1786032187700688 us; error 42 us; skew 500 ppm
I20260806 16:03:07.702735 12576 webserver.cc:460] Webserver started at http://127.25.31.190:35279/ using document root <none> and password file <none>
I20260806 16:03:07.703316 12576 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:07.703380 12576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:07.703613 12576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:07.705715 12576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "ec8e74cc648c4b81b8407dcb53942b9e"
format_stamp: "Formatted at 2026-08-06 16:03:07 on dist-test-slave-tm1g"
server_key: "638e1a872add5ee79634cfcf6eafb7c7"
server_key_iv: "2cf4f032dd135628f7ccd0751f9363fa"
server_key_version: "encryptionkey@0"
I20260806 16:03:07.706293 12576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "ec8e74cc648c4b81b8407dcb53942b9e"
format_stamp: "Formatted at 2026-08-06 16:03:07 on dist-test-slave-tm1g"
server_key: "638e1a872add5ee79634cfcf6eafb7c7"
server_key_iv: "2cf4f032dd135628f7ccd0751f9363fa"
server_key_version: "encryptionkey@0"
I20260806 16:03:07.709968 12576 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:03:07.712441 12590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:07.713712 12576 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:03:07.713856 12576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "ec8e74cc648c4b81b8407dcb53942b9e"
format_stamp: "Formatted at 2026-08-06 16:03:07 on dist-test-slave-tm1g"
server_key: "638e1a872add5ee79634cfcf6eafb7c7"
server_key_iv: "2cf4f032dd135628f7ccd0751f9363fa"
server_key_version: "encryptionkey@0"
I20260806 16:03:07.713970 12576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:07.727430 12576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:07.728097 12576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:07.728294 12576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:07.735273 12576 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:42747
I20260806 16:03:07.735281 12642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:42747 every 8 connection(s)
I20260806 16:03:07.736191 12576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:07.739238 12643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:07.743366 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 12576
I20260806 16:03:07.743508 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:07.743831 25726 external_mini_cluster.cc:1428] Setting key 49a430ad00f774cdbc1ee5e544859ded
I20260806 16:03:07.746125 12643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e: Bootstrap starting.
I20260806 16:03:07.748963 12643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:07.749888 12643 log.cc:824] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:07.751801 12643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e: No bootstrap required, opened a new log
I20260806 16:03:07.754830 12646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.745768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42376 (local address 127.25.31.190:42747)
0806 16:03:07.746505 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:07.746529 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:07.746577 (+ 48us) server_negotiation.cc:407] Connection header received
0806 16:03:07.747628 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:07.747658 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:07.748141 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:07.748534 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:07.749007 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.750631 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.751502 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.752059 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.752229 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.753111 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:07.753116 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:07.753128 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:07.753488 (+ 360us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:07.753543 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:07.753564 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:07.753674 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:07.753791 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 16:03:07.754047 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":125,"threads_started":1}
I20260806 16:03:07.755488 12643 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8e74cc648c4b81b8407dcb53942b9e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42747 } }
I20260806 16:03:07.755728 12643 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:07.755805 12643 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8e74cc648c4b81b8407dcb53942b9e, State: Initialized, Role: FOLLOWER
I20260806 16:03:07.756259 12643 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8e74cc648c4b81b8407dcb53942b9e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42747 } }
I20260806 16:03:07.756404 12643 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:07.756488 12643 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:07.756597 12643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:07.757550 12643 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8e74cc648c4b81b8407dcb53942b9e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42747 } }
I20260806 16:03:07.757875 12643 leader_election.cc:302] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec8e74cc648c4b81b8407dcb53942b9e; no voters:
I20260806 16:03:07.758167 12643 leader_election.cc:288] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:07.758323 12648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:07.758580 12648 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [term 1 LEADER]: Becoming Leader. State: Replica: ec8e74cc648c4b81b8407dcb53942b9e, State: Running, Role: LEADER
I20260806 16:03:07.758874 12648 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8e74cc648c4b81b8407dcb53942b9e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42747 } }
I20260806 16:03:07.759374 12643 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:07.760603 12649 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec8e74cc648c4b81b8407dcb53942b9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8e74cc648c4b81b8407dcb53942b9e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42747 } } }
I20260806 16:03:07.760733 12649 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:07.761049 12650 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec8e74cc648c4b81b8407dcb53942b9e. Latest consensus state: current_term: 1 leader_uuid: "ec8e74cc648c4b81b8407dcb53942b9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8e74cc648c4b81b8407dcb53942b9e" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 42747 } } }
I20260806 16:03:07.761168 12650 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:07.761108 12657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:07.763967 12657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:07.769711 12657 catalog_manager.cc:1347] Generated new cluster ID: 8084bd2169834497bb998c5a99a3c446
I20260806 16:03:07.769783 12657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:07.780793 12657 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:07.781617 12657 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:07.787874 12657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e: Generated new TSK 0
I20260806 16:03:07.788462 12657 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:07.792129 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42747
--builtin_ntp_servers=127.25.31.148:46111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:03:07.901427 12667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:07.901661 12667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:07.901697 12667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:07.905678 12667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:07.905725 12667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:07.905820 12667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:03:07.910653 12667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:07.911679 12667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:07.913007 12667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:07.922367 12677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:07.922431 12667 server_base.cc:1034] running on GCE node
W20260806 16:03:07.922457 12675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:07.922367 12673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:07.923025 12667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:07.923557 12667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:07.924746 12667 hybrid_clock.cc:648] HybridClock initialized: now 1786032187924731 us; error 35 us; skew 500 ppm
I20260806 16:03:07.926625 12667 webserver.cc:460] Webserver started at http://127.25.31.129:36909/ using document root <none> and password file <none>
I20260806 16:03:07.927179 12667 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:07.927273 12667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:07.927500 12667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:07.929442 12667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "611c3f16d49c441f88415ff8d239e28c"
format_stamp: "Formatted at 2026-08-06 16:03:07 on dist-test-slave-tm1g"
server_key: "06111f6afd70e3a43b490b0b6b06201f"
server_key_iv: "e7fe249e4cc37fe5c1ac425e7dedded2"
server_key_version: "encryptionkey@0"
I20260806 16:03:07.930001 12667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "611c3f16d49c441f88415ff8d239e28c"
format_stamp: "Formatted at 2026-08-06 16:03:07 on dist-test-slave-tm1g"
server_key: "06111f6afd70e3a43b490b0b6b06201f"
server_key_iv: "e7fe249e4cc37fe5c1ac425e7dedded2"
server_key_version: "encryptionkey@0"
I20260806 16:03:07.933480 12667 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 16:03:07.935837 12683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:07.937023 12667 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:03:07.937186 12667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "611c3f16d49c441f88415ff8d239e28c"
format_stamp: "Formatted at 2026-08-06 16:03:07 on dist-test-slave-tm1g"
server_key: "06111f6afd70e3a43b490b0b6b06201f"
server_key_iv: "e7fe249e4cc37fe5c1ac425e7dedded2"
server_key_version: "encryptionkey@0"
I20260806 16:03:07.937300 12667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:07.946143 12667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:07.946806 12667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:07.947002 12667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:07.947537 12667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:07.948414 12667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:07.948485 12667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:07.948549 12667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:07.948591 12667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:07.957204 12667 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:43461
I20260806 16:03:07.957252 12796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:43461 every 8 connection(s)
I20260806 16:03:07.958056 12667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 16:03:07.967312 12646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.959832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:33915 (local address 127.25.31.190:42747)
0806 16:03:07.959970 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:07.959973 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:07.961152 (+ 1179us) server_negotiation.cc:407] Connection header received
0806 16:03:07.962684 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:07.962687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:07.962757 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:07.962864 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:07.963711 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.964270 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.965405 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.965643 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.965755 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.966796 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:07.966800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:07.966803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:07.966816 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:07.966844 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:07.966852 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:07.966947 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:07.967102 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 16:03:07.967170 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 16:03:07.968133 12799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:07.960236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42747 (local address 127.25.31.129:33915)
0806 16:03:07.961006 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:07.961046 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:07.962476 (+ 1430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:07.963025 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:07.963031 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:07.963111 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:07.963566 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.963578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.964400 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.964412 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.965261 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:07.965270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:07.965797 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:07.965801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:07.965967 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:07.966546 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:03:07.966547 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:07.966584 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:03:07.966642 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:07.966951 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:07.966962 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:07.966971 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 16:03:07.967102 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 16:03:07.967367 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":156,"threads_started":1}
I20260806 16:03:07.968297 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 12667
I20260806 16:03:07.968441 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 16:03:07.968791 25726 external_mini_cluster.cc:1428] Setting key 2c3b3540d75ac98e11632121412c0a35
I20260806 16:03:07.969334 12797 heartbeater.cc:347] Connected to a master server at 127.25.31.190:42747
I20260806 16:03:07.969558 12797 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:07.970038 12797 heartbeater.cc:511] Master 127.25.31.190:42747 requested a full tablet report, sending...
I20260806 16:03:07.970702 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42747
--builtin_ntp_servers=127.25.31.148:46111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:03:07.971626 12607 ts_manager.cc:194] Registered new tserver with Master: 611c3f16d49c441f88415ff8d239e28c (127.25.31.129:43461)
I20260806 16:03:07.972882 12607 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.31.129:33915
W20260806 16:03:08.080379 12800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:08.080621 12800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:08.080657 12800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:08.084615 12800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:08.084663 12800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:08.084715 12800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:03:08.089442 12800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:08.090474 12800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:08.091799 12800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:08.101415 12806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:08.101426 12805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:08.101504 12800 server_base.cc:1034] running on GCE node
W20260806 16:03:08.101657 12808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:08.102057 12800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:08.102725 12800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:08.103906 12800 hybrid_clock.cc:648] HybridClock initialized: now 1786032188103894 us; error 26 us; skew 500 ppm
I20260806 16:03:08.105758 12800 webserver.cc:460] Webserver started at http://127.25.31.130:33247/ using document root <none> and password file <none>
I20260806 16:03:08.106393 12800 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:08.106491 12800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:08.106704 12800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:08.108702 12800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "8dfb176495a1454bad45a9c5eba76e0a"
format_stamp: "Formatted at 2026-08-06 16:03:08 on dist-test-slave-tm1g"
server_key: "013c202c4365b3efb1565933cec57a98"
server_key_iv: "cfb8bde41e566aca28705d64bea96f64"
server_key_version: "encryptionkey@0"
I20260806 16:03:08.109263 12800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "8dfb176495a1454bad45a9c5eba76e0a"
format_stamp: "Formatted at 2026-08-06 16:03:08 on dist-test-slave-tm1g"
server_key: "013c202c4365b3efb1565933cec57a98"
server_key_iv: "cfb8bde41e566aca28705d64bea96f64"
server_key_version: "encryptionkey@0"
I20260806 16:03:08.112869 12800 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 16:03:08.115265 12814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:08.116410 12800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:03:08.116555 12800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "8dfb176495a1454bad45a9c5eba76e0a"
format_stamp: "Formatted at 2026-08-06 16:03:08 on dist-test-slave-tm1g"
server_key: "013c202c4365b3efb1565933cec57a98"
server_key_iv: "cfb8bde41e566aca28705d64bea96f64"
server_key_version: "encryptionkey@0"
I20260806 16:03:08.116693 12800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:08.131898 12800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:08.132548 12800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:08.132740 12800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:08.133301 12800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:08.134246 12800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:08.134335 12800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:08.134411 12800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:08.134430 12800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:08.143730 12800 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:44923
I20260806 16:03:08.143759 12929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:44923 every 8 connection(s)
I20260806 16:03:08.144675 12800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:03:08.147274 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 12800
I20260806 16:03:08.147403 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 16:03:08.147729 25726 external_mini_cluster.cc:1428] Setting key 2b160a06694f99c59b7c7319e4ef50b2
I20260806 16:03:08.149715 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42747
--builtin_ntp_servers=127.25.31.148:46111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:03:08.154654 12646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.146516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:45257 (local address 127.25.31.190:42747)
0806 16:03:08.146659 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:08.146663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:08.147872 (+ 1209us) server_negotiation.cc:407] Connection header received
0806 16:03:08.149517 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:08.149520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:08.149575 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:08.149677 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:08.150676 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.151212 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.152350 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.152741 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.152864 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.154003 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:08.154010 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:08.154017 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:08.154035 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:08.154074 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:08.154094 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:08.154184 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:08.154353 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 16:03:08.154441 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:03:08.155416 12932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.146927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42747 (local address 127.25.31.130:45257)
0806 16:03:08.147705 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:08.147748 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:08.149313 (+ 1565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:08.149870 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:08.149879 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:08.149997 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:08.150533 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:08.150547 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.151343 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.151353 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:08.152219 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:08.152228 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.152900 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.152902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:08.153051 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.153679 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:03:08.153681 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:08.153742 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:03:08.153828 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:08.154189 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:08.154196 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:08.154200 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 16:03:08.154293 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 16:03:08.154570 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":161,"threads_started":1}
I20260806 16:03:08.156621 12930 heartbeater.cc:347] Connected to a master server at 127.25.31.190:42747
I20260806 16:03:08.156900 12930 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:08.157573 12930 heartbeater.cc:511] Master 127.25.31.190:42747 requested a full tablet report, sending...
I20260806 16:03:08.158769 12607 ts_manager.cc:194] Registered new tserver with Master: 8dfb176495a1454bad45a9c5eba76e0a (127.25.31.130:44923)
I20260806 16:03:08.159452 12607 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.31.130:45257
W20260806 16:03:08.261873 12933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:08.262159 12933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:08.262225 12933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:08.266247 12933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:08.266350 12933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:08.266443 12933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:03:08.271237 12933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:46111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:42747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:08.272326 12933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:08.273710 12933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 16:03:08.283421 12933 server_base.cc:1034] running on GCE node
W20260806 16:03:08.283370 12939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:08.283370 12938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:08.283387 12941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:08.284052 12933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:08.284672 12933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:08.285847 12933 hybrid_clock.cc:648] HybridClock initialized: now 1786032188285821 us; error 40 us; skew 500 ppm
I20260806 16:03:08.287667 12933 webserver.cc:460] Webserver started at http://127.25.31.131:38911/ using document root <none> and password file <none>
I20260806 16:03:08.288259 12933 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:08.288352 12933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:08.288586 12933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:08.290663 12933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "96a124d295764ab98dadb892d5f09b95"
format_stamp: "Formatted at 2026-08-06 16:03:08 on dist-test-slave-tm1g"
server_key: "7ac6e34197df428ed1d57ef4eb4baa8e"
server_key_iv: "d3a2b78102f212ba7ed6f06be94f88da"
server_key_version: "encryptionkey@0"
I20260806 16:03:08.291231 12933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "96a124d295764ab98dadb892d5f09b95"
format_stamp: "Formatted at 2026-08-06 16:03:08 on dist-test-slave-tm1g"
server_key: "7ac6e34197df428ed1d57ef4eb4baa8e"
server_key_iv: "d3a2b78102f212ba7ed6f06be94f88da"
server_key_version: "encryptionkey@0"
I20260806 16:03:08.295077 12933 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 16:03:08.297571 12947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:08.298831 12933 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:03:08.298964 12933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "96a124d295764ab98dadb892d5f09b95"
format_stamp: "Formatted at 2026-08-06 16:03:08 on dist-test-slave-tm1g"
server_key: "7ac6e34197df428ed1d57ef4eb4baa8e"
server_key_iv: "d3a2b78102f212ba7ed6f06be94f88da"
server_key_version: "encryptionkey@0"
I20260806 16:03:08.299075 12933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:08.321435 12933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:08.322111 12933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:08.322331 12933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:08.322917 12933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:08.323846 12933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:08.323894 12933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:08.323958 12933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:08.324008 12933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:08.332945 12933 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:36703
I20260806 16:03:08.332983 13062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:36703 every 8 connection(s)
I20260806 16:03:08.333809 12933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:03:08.336721 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 12933
I20260806 16:03:08.336835 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 16:03:08.337172 25726 external_mini_cluster.cc:1428] Setting key 50ecc96bbdf568a4fbff54dec16180a4
I20260806 16:03:08.343417 12646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.335576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:59761 (local address 127.25.31.190:42747)
0806 16:03:08.335734 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:08.335739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:08.336924 (+ 1185us) server_negotiation.cc:407] Connection header received
0806 16:03:08.338774 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:08.338777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:08.338831 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:08.338916 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:08.339879 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.340427 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.341593 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.341848 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.341927 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.343027 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:08.343030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:08.343032 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:08.343042 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:08.343061 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:08.343068 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:08.343124 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:08.343260 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 16:03:08.343308 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 16:03:08.344209 13065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.336011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:42747 (local address 127.25.31.131:59761)
0806 16:03:08.336755 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:08.336814 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:08.338548 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:08.339063 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:08.339070 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:08.339188 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:08.339749 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:08.339762 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.340579 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.340585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:08.341458 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:08.341467 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.341985 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.341988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:08.342122 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.342799 (+ 677us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:03:08.342801 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:08.342839 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:03:08.342901 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:08.343140 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:08.343147 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:08.343151 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 16:03:08.343242 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 16:03:08.343456 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":115,"threads_started":1}
I20260806 16:03:08.345285 13063 heartbeater.cc:347] Connected to a master server at 127.25.31.190:42747
I20260806 16:03:08.345512 13063 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:08.345986 13063 heartbeater.cc:511] Master 127.25.31.190:42747 requested a full tablet report, sending...
I20260806 16:03:08.347034 12607 ts_manager.cc:194] Registered new tserver with Master: 96a124d295764ab98dadb892d5f09b95 (127.25.31.131:36703)
I20260806 16:03:08.347554 12607 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.31.131:59761
I20260806 16:03:08.350783 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:03:08.356284 12646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.352675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42386 (local address 127.25.31.190:42747)
0806 16:03:08.352809 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:08.352814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:08.353005 (+ 191us) server_negotiation.cc:407] Connection header received
0806 16:03:08.353077 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:08.353079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:08.353115 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:08.353172 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:08.353381 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.353867 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.354751 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.354952 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.355028 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.355822 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:08.355824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:08.355826 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:08.355836 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:08.355851 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:08.355858 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:08.355943 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:08.356071 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 16:03:08.356129 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 16:03:08.358778 12607 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:03:08.362053 12607 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:03:08.375931 13075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.369999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:44923 (local address 127.0.0.1:42568)
0806 16:03:08.370714 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:08.370763 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:08.370879 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:08.371698 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:08.371706 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:08.371734 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:08.371889 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:08.371898 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.373100 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.373105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:08.374419 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:08.374429 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.375367 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.375371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:08.375477 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.375517 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 16:03:08.375638 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 16:03:08.375763 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":5,"thread_start_us":79,"threads_started":1}
I20260806 16:03:08.375963 13076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.370145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42568 (local address 127.25.31.130:44923)
0806 16:03:08.371168 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:08.371183 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:08.371207 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:03:08.371283 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:08.371288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:08.371510 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:08.372040 (+ 530us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:08.372192 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.372988 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.374538 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.375245 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.375356 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.375561 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:08.375672 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 16:03:08.375783 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":158,"threads_started":1}
I20260806 16:03:08.376204 13074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.370185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:36703 (local address 127.0.0.1:55990)
0806 16:03:08.371109 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:08.371125 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:08.371204 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:08.371912 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:08.371916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:08.371937 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:08.372034 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:08.372039 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.373464 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.373467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:08.374727 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:08.374738 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.375767 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.375771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:08.375863 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.375902 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 16:03:08.375952 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 16:03:08.376040 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":18,"thread_start_us":128,"threads_started":1}
I20260806 16:03:08.376762 13078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.370843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:43461 (local address 127.0.0.1:52248)
0806 16:03:08.371682 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:08.371695 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:08.371774 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:08.372190 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:08.372193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:08.372208 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:08.372281 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:08.372287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.373742 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.373745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:08.374918 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:08.374925 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.376363 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.376377 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:08.376457 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.376476 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 16:03:08.376564 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 16:03:08.376638 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":48,"thread_start_us":108,"threads_started":1}
I20260806 16:03:08.376842 13077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.370762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55990 (local address 127.25.31.131:36703)
0806 16:03:08.371460 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:08.371474 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:08.371497 (+ 23us) server_negotiation.cc:407] Connection header received
0806 16:03:08.371579 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:08.371584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:08.371793 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:08.371948 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:08.372123 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.373316 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.374858 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.375631 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.376387 (+ 756us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.376561 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:08.376608 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 16:03:08.376690 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":169,"threads_started":1}
I20260806 16:03:08.379009 12863 tablet_service.cc:1467] Processing CreateTablet for tablet c85aa5a674024f66bcf4649f4a03e172 (DEFAULT_TABLE table=test-table [id=eeb3227c2e2845469a83e06cf4390b51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:08.379140 12996 tablet_service.cc:1467] Processing CreateTablet for tablet c85aa5a674024f66bcf4649f4a03e172 (DEFAULT_TABLE table=test-table [id=eeb3227c2e2845469a83e06cf4390b51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:08.379984 12996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85aa5a674024f66bcf4649f4a03e172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:08.379984 12863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85aa5a674024f66bcf4649f4a03e172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:08.376883 13079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.371284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52248 (local address 127.25.31.129:43461)
0806 16:03:08.371636 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:08.371646 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:08.371778 (+ 132us) server_negotiation.cc:407] Connection header received
0806 16:03:08.371890 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:08.371898 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:08.372073 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:08.372220 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:08.372366 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.373624 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.375028 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.375731 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.375826 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.376002 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:08.376613 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 16:03:08.376709 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":138,"threads_started":1}
I20260806 16:03:08.387050 13080 tablet_bootstrap.cc:492] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a: Bootstrap starting.
I20260806 16:03:08.386757 12731 tablet_service.cc:1467] Processing CreateTablet for tablet c85aa5a674024f66bcf4649f4a03e172 (DEFAULT_TABLE table=test-table [id=eeb3227c2e2845469a83e06cf4390b51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:08.387467 13081 tablet_bootstrap.cc:492] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95: Bootstrap starting.
I20260806 16:03:08.387696 12731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85aa5a674024f66bcf4649f4a03e172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:08.389309 13080 tablet_bootstrap.cc:654] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:08.389309 13081 tablet_bootstrap.cc:654] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:08.390183 13080 log.cc:824] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:08.390183 13081 log.cc:824] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:08.392087 13080 tablet_bootstrap.cc:492] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a: No bootstrap required, opened a new log
I20260806 16:03:08.392089 13081 tablet_bootstrap.cc:492] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95: No bootstrap required, opened a new log
I20260806 16:03:08.392285 13080 ts_tablet_manager.cc:1397] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 16:03:08.392438 13081 ts_tablet_manager.cc:1397] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 16:03:08.393705 13084 tablet_bootstrap.cc:492] T c85aa5a674024f66bcf4649f4a03e172 P 611c3f16d49c441f88415ff8d239e28c: Bootstrap starting.
I20260806 16:03:08.396065 13081 raft_consensus.cc:348] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfb176495a1454bad45a9c5eba76e0a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 44923 } } peers { permanent_uuid: "96a124d295764ab98dadb892d5f09b95" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36703 } } peers { permanent_uuid: "611c3f16d49c441f88415ff8d239e28c" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43461 } }
I20260806 16:03:08.396236 13084 tablet_bootstrap.cc:654] T c85aa5a674024f66bcf4649f4a03e172 P 611c3f16d49c441f88415ff8d239e28c: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:08.396298 13081 raft_consensus.cc:374] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:08.396359 13081 raft_consensus.cc:729] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a124d295764ab98dadb892d5f09b95, State: Initialized, Role: FOLLOWER
I20260806 16:03:08.396431 13080 raft_consensus.cc:348] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfb176495a1454bad45a9c5eba76e0a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 44923 } } peers { permanent_uuid: "96a124d295764ab98dadb892d5f09b95" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36703 } } peers { permanent_uuid: "611c3f16d49c441f88415ff8d239e28c" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43461 } }
I20260806 16:03:08.396687 13080 raft_consensus.cc:374] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:08.396749 13080 raft_consensus.cc:729] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dfb176495a1454bad45a9c5eba76e0a, State: Initialized, Role: FOLLOWER
I20260806 16:03:08.396838 13081 consensus_queue.cc:260] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfb176495a1454bad45a9c5eba76e0a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 44923 } } peers { permanent_uuid: "96a124d295764ab98dadb892d5f09b95" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36703 } } peers { permanent_uuid: "611c3f16d49c441f88415ff8d239e28c" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43461 } }
I20260806 16:03:08.397177 13084 log.cc:824] T c85aa5a674024f66bcf4649f4a03e172 P 611c3f16d49c441f88415ff8d239e28c: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:08.397161 13080 consensus_queue.cc:260] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfb176495a1454bad45a9c5eba76e0a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 44923 } } peers { permanent_uuid: "96a124d295764ab98dadb892d5f09b95" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36703 } } peers { permanent_uuid: "611c3f16d49c441f88415ff8d239e28c" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43461 } }
I20260806 16:03:08.397533 13063 heartbeater.cc:503] Master 127.25.31.190:42747 was elected leader, sending a full tablet report...
I20260806 16:03:08.397815 13081 ts_tablet_manager.cc:1428] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 16:03:08.398101 13080 ts_tablet_manager.cc:1428] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 16:03:08.398463 12930 heartbeater.cc:503] Master 127.25.31.190:42747 was elected leader, sending a full tablet report...
I20260806 16:03:08.400033 13084 tablet_bootstrap.cc:492] T c85aa5a674024f66bcf4649f4a03e172 P 611c3f16d49c441f88415ff8d239e28c: No bootstrap required, opened a new log
I20260806 16:03:08.400259 13084 ts_tablet_manager.cc:1397] T c85aa5a674024f66bcf4649f4a03e172 P 611c3f16d49c441f88415ff8d239e28c: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 16:03:08.403709 13084 raft_consensus.cc:348] T c85aa5a674024f66bcf4649f4a03e172 P 611c3f16d49c441f88415ff8d239e28c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfb176495a1454bad45a9c5eba76e0a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 44923 } } peers { permanent_uuid: "96a124d295764ab98dadb892d5f09b95" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36703 } } peers { permanent_uuid: "611c3f16d49c441f88415ff8d239e28c" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43461 } }
I20260806 16:03:08.403898 13084 raft_consensus.cc:374] T c85aa5a674024f66bcf4649f4a03e172 P 611c3f16d49c441f88415ff8d239e28c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:08.403970 13084 raft_consensus.cc:729] T c85aa5a674024f66bcf4649f4a03e172 P 611c3f16d49c441f88415ff8d239e28c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611c3f16d49c441f88415ff8d239e28c, State: Initialized, Role: FOLLOWER
I20260806 16:03:08.404359 13084 consensus_queue.cc:260] T c85aa5a674024f66bcf4649f4a03e172 P 611c3f16d49c441f88415ff8d239e28c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfb176495a1454bad45a9c5eba76e0a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 44923 } } peers { permanent_uuid: "96a124d295764ab98dadb892d5f09b95" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36703 } } peers { permanent_uuid: "611c3f16d49c441f88415ff8d239e28c" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43461 } }
I20260806 16:03:08.404963 12797 heartbeater.cc:503] Master 127.25.31.190:42747 was elected leader, sending a full tablet report...
I20260806 16:03:08.405165 13084 ts_tablet_manager.cc:1428] T c85aa5a674024f66bcf4649f4a03e172 P 611c3f16d49c441f88415ff8d239e28c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260806 16:03:08.459236 12798 tablet.cc:2369] T c85aa5a674024f66bcf4649f4a03e172 P 611c3f16d49c441f88415ff8d239e28c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:03:08.584834 13064 tablet.cc:2369] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:03:08.605926 13086 raft_consensus.cc:482] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:03:08.606177 13086 raft_consensus.cc:504] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfb176495a1454bad45a9c5eba76e0a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 44923 } } peers { permanent_uuid: "96a124d295764ab98dadb892d5f09b95" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36703 } } peers { permanent_uuid: "611c3f16d49c441f88415ff8d239e28c" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43461 } }
I20260806 16:03:08.607429 13086 leader_election.cc:288] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a124d295764ab98dadb892d5f09b95 (127.25.31.131:36703), 611c3f16d49c441f88415ff8d239e28c (127.25.31.129:43461)
I20260806 16:03:08.611340 13077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.607841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:45973 (local address 127.25.31.131:36703)
0806 16:03:08.607976 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:08.607981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:08.608010 (+ 29us) server_negotiation.cc:407] Connection header received
0806 16:03:08.608069 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:08.608072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:08.608119 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:08.608213 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:08.608582 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.609179 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.610331 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.610762 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.610894 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.610946 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:08.611128 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 16:03:08.611204 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 16:03:08.611371 13089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.607708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:43461 (local address 127.25.31.130:43393)
0806 16:03:08.608124 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:08.608138 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:08.608232 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:08.608553 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:08.608555 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:08.608576 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:08.608694 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:08.608700 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.609467 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.609469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:08.610257 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:08.610270 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.610931 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.610935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:08.611034 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.611051 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 16:03:08.611148 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 16:03:08.611212 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":115,"threads_started":1}
I20260806 16:03:08.611373 13079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.607848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:43393 (local address 127.25.31.129:43461)
0806 16:03:08.607976 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:08.607981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:08.608257 (+ 276us) server_negotiation.cc:407] Connection header received
0806 16:03:08.608322 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:08.608325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:08.608411 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:08.608509 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:08.608814 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.609354 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.610404 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.610777 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.610918 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.610969 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:08.611186 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 16:03:08.611263 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 16:03:08.611373 12932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.607616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:36703 (local address 127.25.31.130:45973)
0806 16:03:08.607748 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:08.607767 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:08.607861 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:08.608277 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:08.608282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:08.608306 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:08.608451 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:08.608463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.609292 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.609295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:08.610166 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:08.610176 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.610886 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.610890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:08.610993 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.611016 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 16:03:08.611086 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 16:03:08.611177 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 16:03:08.612138 12751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85aa5a674024f66bcf4649f4a03e172" candidate_uuid: "8dfb176495a1454bad45a9c5eba76e0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611c3f16d49c441f88415ff8d239e28c" is_pre_election: true
I20260806 16:03:08.612191 13016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85aa5a674024f66bcf4649f4a03e172" candidate_uuid: "8dfb176495a1454bad45a9c5eba76e0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a124d295764ab98dadb892d5f09b95" is_pre_election: true
I20260806 16:03:08.612444 12751 raft_consensus.cc:2454] T c85aa5a674024f66bcf4649f4a03e172 P 611c3f16d49c441f88415ff8d239e28c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dfb176495a1454bad45a9c5eba76e0a in term 0.
I20260806 16:03:08.612474 13016 raft_consensus.cc:2454] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8dfb176495a1454bad45a9c5eba76e0a in term 0.
I20260806 16:03:08.612982 12817 leader_election.cc:302] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611c3f16d49c441f88415ff8d239e28c, 8dfb176495a1454bad45a9c5eba76e0a; no voters:
I20260806 16:03:08.613269 13086 raft_consensus.cc:2781] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:03:08.613363 13086 raft_consensus.cc:482] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:03:08.613395 13086 raft_consensus.cc:3037] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:08.614514 13086 raft_consensus.cc:504] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfb176495a1454bad45a9c5eba76e0a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 44923 } } peers { permanent_uuid: "96a124d295764ab98dadb892d5f09b95" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36703 } } peers { permanent_uuid: "611c3f16d49c441f88415ff8d239e28c" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43461 } }
I20260806 16:03:08.614900 13086 leader_election.cc:288] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [CANDIDATE]: Term 1 election: Requested vote from peers 96a124d295764ab98dadb892d5f09b95 (127.25.31.131:36703), 611c3f16d49c441f88415ff8d239e28c (127.25.31.129:43461)
I20260806 16:03:08.615258 13016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85aa5a674024f66bcf4649f4a03e172" candidate_uuid: "8dfb176495a1454bad45a9c5eba76e0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a124d295764ab98dadb892d5f09b95"
I20260806 16:03:08.615389 13016 raft_consensus.cc:3037] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:08.616426 13016 raft_consensus.cc:2454] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dfb176495a1454bad45a9c5eba76e0a in term 1.
I20260806 16:03:08.616773 12751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85aa5a674024f66bcf4649f4a03e172" candidate_uuid: "8dfb176495a1454bad45a9c5eba76e0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611c3f16d49c441f88415ff8d239e28c"
I20260806 16:03:08.616828 12816 leader_election.cc:302] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dfb176495a1454bad45a9c5eba76e0a, 96a124d295764ab98dadb892d5f09b95; no voters:
I20260806 16:03:08.616925 12751 raft_consensus.cc:3037] T c85aa5a674024f66bcf4649f4a03e172 P 611c3f16d49c441f88415ff8d239e28c [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:08.617007 13086 raft_consensus.cc:2781] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:08.617228 13086 raft_consensus.cc:686] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [term 1 LEADER]: Becoming Leader. State: Replica: 8dfb176495a1454bad45a9c5eba76e0a, State: Running, Role: LEADER
I20260806 16:03:08.617578 13086 consensus_queue.cc:237] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfb176495a1454bad45a9c5eba76e0a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 44923 } } peers { permanent_uuid: "96a124d295764ab98dadb892d5f09b95" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36703 } } peers { permanent_uuid: "611c3f16d49c441f88415ff8d239e28c" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43461 } }
I20260806 16:03:08.618149 12751 raft_consensus.cc:2454] T c85aa5a674024f66bcf4649f4a03e172 P 611c3f16d49c441f88415ff8d239e28c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dfb176495a1454bad45a9c5eba76e0a in term 1.
I20260806 16:03:08.620976 12606 catalog_manager.cc:5495] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8dfb176495a1454bad45a9c5eba76e0a (127.25.31.130). New cstate: current_term: 1 leader_uuid: "8dfb176495a1454bad45a9c5eba76e0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dfb176495a1454bad45a9c5eba76e0a" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 44923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a124d295764ab98dadb892d5f09b95" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611c3f16d49c441f88415ff8d239e28c" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43461 } health_report { overall_health: UNKNOWN } } }
W20260806 16:03:08.645850 12931 tablet.cc:2369] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:03:08.669070 13076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.665717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42572 (local address 127.25.31.130:44923)
0806 16:03:08.665847 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:08.665850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:08.665863 (+ 13us) server_negotiation.cc:407] Connection header received
0806 16:03:08.665931 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:08.665933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:08.665986 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:08.666076 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:08.666340 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.666850 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.667741 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:08.667962 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:08.668060 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:08.668325 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:08.668717 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:08.668830 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:08.668910 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 16:03:08.668944 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 16:03:08.674811 13016 raft_consensus.cc:1261] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95 [term 1 FOLLOWER]: Refusing update from remote peer 8dfb176495a1454bad45a9c5eba76e0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:08.674811 12751 raft_consensus.cc:1261] T c85aa5a674024f66bcf4649f4a03e172 P 611c3f16d49c441f88415ff8d239e28c [term 1 FOLLOWER]: Refusing update from remote peer 8dfb176495a1454bad45a9c5eba76e0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:08.675556 13090 consensus_queue.cc:1035] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a124d295764ab98dadb892d5f09b95" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 36703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:03:08.675743 13086 consensus_queue.cc:1035] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "611c3f16d49c441f88415ff8d239e28c" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 43461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:03:08.683171 13094 mvcc.cc:205] Tried to move back new op lower bound from 7315587844804620288 to 7315587844581187584. Current Snapshot: MvccSnapshot[applied={T|T < 7315587844804620288}]
I20260806 16:03:08.690852 12606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:03:08.691035 12606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:03:08.693586 12606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ec8e74cc648c4b81b8407dcb53942b9e: Sending DeleteTablet for 3 replicas of tablet c85aa5a674024f66bcf4649f4a03e172
I20260806 16:03:08.694365 12863 tablet_service.cc:1514] Processing DeleteTablet for tablet c85aa5a674024f66bcf4649f4a03e172 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:08 UTC) from {username='slave'} at 127.0.0.1:42568
I20260806 16:03:08.694557 12996 tablet_service.cc:1514] Processing DeleteTablet for tablet c85aa5a674024f66bcf4649f4a03e172 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:08 UTC) from {username='slave'} at 127.0.0.1:55990
I20260806 16:03:08.694572 12731 tablet_service.cc:1514] Processing DeleteTablet for tablet c85aa5a674024f66bcf4649f4a03e172 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:08 UTC) from {username='slave'} at 127.0.0.1:52248
I20260806 16:03:08.695053 13103 tablet_replica.cc:330] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a: stopping tablet replica
I20260806 16:03:08.695353 13104 tablet_replica.cc:330] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95: stopping tablet replica
I20260806 16:03:08.695701 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 12667
I20260806 16:03:08.695698 13104 raft_consensus.cc:2229] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:03:08.699334 13104 raft_consensus.cc:2258] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:03:08.701009 13104 ts_tablet_manager.cc:1905] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:03:08.702638 13103 raft_consensus.cc:2229] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:03:08.703019 13103 raft_consensus.cc:2258] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:03:08.704512 13104 ts_tablet_manager.cc:1918] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:03:08.704614 13104 log.cc:1196] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/c85aa5a674024f66bcf4649f4a03e172
I20260806 16:03:08.705060 13104 ts_tablet_manager.cc:1939] T c85aa5a674024f66bcf4649f4a03e172 P 96a124d295764ab98dadb892d5f09b95: Deleting consensus metadata
I20260806 16:03:08.705175 13103 ts_tablet_manager.cc:1905] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:03:08.706391 12592 catalog_manager.cc:4841] TS 96a124d295764ab98dadb892d5f09b95 (127.25.31.131:36703): tablet c85aa5a674024f66bcf4649f4a03e172 (table test-table [id=eeb3227c2e2845469a83e06cf4390b51]) successfully deleted
W20260806 16:03:08.706970 12593 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:43461 (error 108)
I20260806 16:03:08.707950 13103 ts_tablet_manager.cc:1918] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:03:08.708019 13078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.707699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:43461 (local address 127.0.0.1:52264)
0806 16:03:08.707829 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:08.707916 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:43461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 16:03:08.708047 13103 log.cc:1196] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/c85aa5a674024f66bcf4649f4a03e172
I20260806 16:03:08.708123 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 12800
I20260806 16:03:08.708346 13103 ts_tablet_manager.cc:1939] T c85aa5a674024f66bcf4649f4a03e172 P 8dfb176495a1454bad45a9c5eba76e0a: Deleting consensus metadata
W20260806 16:03:08.708343 12593 catalog_manager.cc:4584] TS 611c3f16d49c441f88415ff8d239e28c (127.25.31.129:43461): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c85aa5a674024f66bcf4649f4a03e172: Network error: Client connection negotiation failed: client connection to 127.25.31.129:43461: connect: Connection refused (error 111)
I20260806 16:03:08.714196 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 12933
I20260806 16:03:08.715077 13078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:08.714337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:44923 (local address 127.0.0.1:42576)
0806 16:03:08.714935 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:08.714996 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.130:44923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520}
I20260806 16:03:08.720432 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 12576
2026-08-06T16:03:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1182 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:08 dist-test-slave-tm1g krb5kdc[13109](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:08 dist-test-slave-tm1g krb5kdc[13109](info): set up 2 sockets
Aug 06 16:03:08 dist-test-slave-tm1g krb5kdc[13109](info): commencing operation
krb5kdc: starting...
W20260806 16:03:09.794601 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:09 dist-test-slave-tm1g krb5kdc[13109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:03:09.911178 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:41051
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:41051
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:10.019330 13122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:10.019595 13122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:10.019634 13122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:10.023485 13122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:10.023532 13122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:10.023551 13122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:10.023566 13122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:10.023581 13122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:10.028681 13122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:41051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:41051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:10.029618 13122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:10.030583 13122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:10.038748 13130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:10.038785 13128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:10.038946 13127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:10.038967 13122 server_base.cc:1034] running on GCE node
I20260806 16:03:10.039600 13122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:10.040620 13122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:10.041807 13122 hybrid_clock.cc:648] HybridClock initialized: now 1786032190041793 us; error 34 us; skew 500 ppm
Aug 06 16:03:10 dist-test-slave-tm1g krb5kdc[13109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032190, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:10.049875 13122 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:03:10.050797 13122 webserver.cc:460] Webserver started at http://127.25.31.190:34779/ using document root <none> and password file <none>
I20260806 16:03:10.051312 13122 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:10.051369 13122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:10.051530 13122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:10.053464 13122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "65948732b33d420c9bcc4c2be4270dbd"
format_stamp: "Formatted at 2026-08-06 16:03:10 on dist-test-slave-tm1g"
server_key: "3956b03dca3e71ef556773d00cadc1dd"
server_key_iv: "380c91f08336a0334d7bcceb97eb35af"
server_key_version: "encryptionkey@0"
I20260806 16:03:10.053977 13122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "65948732b33d420c9bcc4c2be4270dbd"
format_stamp: "Formatted at 2026-08-06 16:03:10 on dist-test-slave-tm1g"
server_key: "3956b03dca3e71ef556773d00cadc1dd"
server_key_iv: "380c91f08336a0334d7bcceb97eb35af"
server_key_version: "encryptionkey@0"
I20260806 16:03:10.057518 13122 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 16:03:10.059913 13137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:10.061157 13122 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:03:10.061285 13122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "65948732b33d420c9bcc4c2be4270dbd"
format_stamp: "Formatted at 2026-08-06 16:03:10 on dist-test-slave-tm1g"
server_key: "3956b03dca3e71ef556773d00cadc1dd"
server_key_iv: "380c91f08336a0334d7bcceb97eb35af"
server_key_version: "encryptionkey@0"
I20260806 16:03:10.061388 13122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:10.075695 13122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:10.078470 13122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:10.078661 13122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:10.085531 13122 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:41051
I20260806 16:03:10.085561 13190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:41051 every 8 connection(s)
I20260806 16:03:10.086496 13122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:10.086827 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 13122
I20260806 16:03:10.086990 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:10.087313 25726 external_mini_cluster.cc:1428] Setting key 137c9a17e0145bc57f4d59fa2687ebf7
I20260806 16:03:10.090086 13191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 16:03:10 dist-test-slave-tm1g krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:03:10.096820 13191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd: Bootstrap starting.
I20260806 16:03:10.098958 13191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:10.099689 13191 log.cc:824] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:10.101433 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.088700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34328 (local address 127.25.31.190:41051)
0806 16:03:10.089099 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:10.089123 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:10.089159 (+ 36us) server_negotiation.cc:407] Connection header received
0806 16:03:10.089786 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:10.089801 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:10.090102 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:10.090480 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:10.091338 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.092314 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.093528 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.093977 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.094095 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.096857 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:10.096862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:10.096874 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:10.096901 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:10.098674 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:10.099122 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:10.099132 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:10.099136 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:10.099242 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:10.099541 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:10.099545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:10.099546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:10.099980 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:10.100175 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:10.100466 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 16:03:10.100749 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":140,"threads_started":1}
I20260806 16:03:10.101682 13191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd: No bootstrap required, opened a new log
I20260806 16:03:10.105827 13191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65948732b33d420c9bcc4c2be4270dbd" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41051 } }
I20260806 16:03:10.106065 13191 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:10.106143 13191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65948732b33d420c9bcc4c2be4270dbd, State: Initialized, Role: FOLLOWER
I20260806 16:03:10.106617 13191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65948732b33d420c9bcc4c2be4270dbd" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41051 } }
I20260806 16:03:10.106767 13191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:10.106837 13191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:10.106937 13191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:10.107822 13191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65948732b33d420c9bcc4c2be4270dbd" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41051 } }
I20260806 16:03:10.108153 13191 leader_election.cc:302] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65948732b33d420c9bcc4c2be4270dbd; no voters:
I20260806 16:03:10.108456 13191 leader_election.cc:288] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:10.108567 13196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:10.108842 13196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [term 1 LEADER]: Becoming Leader. State: Replica: 65948732b33d420c9bcc4c2be4270dbd, State: Running, Role: LEADER
I20260806 16:03:10.109179 13196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65948732b33d420c9bcc4c2be4270dbd" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41051 } }
I20260806 16:03:10.109647 13191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:10.110831 13197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65948732b33d420c9bcc4c2be4270dbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65948732b33d420c9bcc4c2be4270dbd" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41051 } } }
I20260806 16:03:10.110963 13197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:10.110787 13198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65948732b33d420c9bcc4c2be4270dbd. Latest consensus state: current_term: 1 leader_uuid: "65948732b33d420c9bcc4c2be4270dbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65948732b33d420c9bcc4c2be4270dbd" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41051 } } }
I20260806 16:03:10.111281 13198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:10.111426 13205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:10.114183 13205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:10.119488 13205 catalog_manager.cc:1347] Generated new cluster ID: c6ec32f1ef4e42588861255b2293aa57
I20260806 16:03:10.119551 13205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:10.137609 13205 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:10.138515 13205 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:10.146261 13205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd: Generated new TSK 0
I20260806 16:03:10.147018 13205 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.31.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
Entry for principal kudu/127.25.31.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab.
I20260806 16:03:10.177387 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:41051
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:10.286765 13217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:10.286998 13217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:10.287066 13217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:10.291028 13217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:10.291078 13217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:10.291163 13217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:03:10.296296 13217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.129.keytab
--principal=kudu/127.25.31.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:41051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:10.297266 13217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:10.298242 13217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:10.307334 13225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:10.307365 13223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:10.307421 13217 server_base.cc:1034] running on GCE node
W20260806 16:03:10.307368 13222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:10.307969 13217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:10.308560 13217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:10.309710 13217 hybrid_clock.cc:648] HybridClock initialized: now 1786032190309684 us; error 39 us; skew 500 ppm
Aug 06 16:03:10 dist-test-slave-tm1g krb5kdc[13109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032190, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:10.312537 13217 init.cc:377] Logged in from keytab as kudu/127.25.31.129@KRBTEST.COM (short username kudu)
I20260806 16:03:10.313297 13217 webserver.cc:460] Webserver started at http://127.25.31.129:33885/ using document root <none> and password file <none>
I20260806 16:03:10.313788 13217 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:10.313845 13217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:10.314006 13217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:10.315953 13217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "2339206e979f4b778925e288b71ba077"
format_stamp: "Formatted at 2026-08-06 16:03:10 on dist-test-slave-tm1g"
server_key: "c46dbda7a6204290cf4a7344c30adcd9"
server_key_iv: "adddb224d4d39e902ef8ee8e179c89c4"
server_key_version: "encryptionkey@0"
I20260806 16:03:10.316442 13217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "2339206e979f4b778925e288b71ba077"
format_stamp: "Formatted at 2026-08-06 16:03:10 on dist-test-slave-tm1g"
server_key: "c46dbda7a6204290cf4a7344c30adcd9"
server_key_iv: "adddb224d4d39e902ef8ee8e179c89c4"
server_key_version: "encryptionkey@0"
I20260806 16:03:10.319970 13217 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 16:03:10.322207 13232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:10.323411 13217 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:03:10.323544 13217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "2339206e979f4b778925e288b71ba077"
format_stamp: "Formatted at 2026-08-06 16:03:10 on dist-test-slave-tm1g"
server_key: "c46dbda7a6204290cf4a7344c30adcd9"
server_key_iv: "adddb224d4d39e902ef8ee8e179c89c4"
server_key_version: "encryptionkey@0"
I20260806 16:03:10.323652 13217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:10.336444 13217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:10.339574 13217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:10.339771 13217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:10.340329 13217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:10.341116 13217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:10.341187 13217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:10.341259 13217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:10.341302 13217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:10.350732 13217 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:38589
I20260806 16:03:10.350769 13346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:38589 every 8 connection(s)
I20260806 16:03:10.351584 13217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 16:03:10.353166 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 13217
I20260806 16:03:10.353319 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 16:03:10.353652 25726 external_mini_cluster.cc:1428] Setting key ee47978d8c0a68bae560596ee920f6f3
Aug 06 16:03:10 dist-test-slave-tm1g krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032190, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.129@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.31.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.130@KRBTEST.COM" created.
I20260806 16:03:10.365906 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.353325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:41501 (local address 127.25.31.190:41051)
0806 16:03:10.353492 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:10.353498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:10.354893 (+ 1395us) server_negotiation.cc:407] Connection header received
0806 16:03:10.355832 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:10.355836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:10.355889 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:10.356001 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:10.357436 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.357986 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.359202 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.359438 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.359550 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.362162 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:10.362164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:10.362166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:10.362190 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:10.363748 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:10.364287 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:10.364289 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:10.364291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:10.364368 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:10.364660 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:10.364662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:10.364663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:10.364824 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:10.364897 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:10.365511 (+ 614us) server_negotiation.cc:299] Negotiation successful
0806 16:03:10.365659 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 16:03:10.366708 13349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.353677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:41051 (local address 127.25.31.129:41501)
0806 16:03:10.354749 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:10.354801 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:10.355633 (+ 832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:10.356179 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:10.356186 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:10.356584 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:10.357273 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.357295 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.358104 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.358106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.359065 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.359075 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.359588 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.359592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.359762 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.360330 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:10.360346 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:10.361979 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:10.363855 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:10.363859 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:10.363870 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:10.364188 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:10.364451 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:10.364453 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:10.364455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:10.364582 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:10.364910 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:10.364915 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:10.365185 (+ 270us) client_negotiation.cc:769] Sending connection context
0806 16:03:10.365352 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 16:03:10.365735 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"thread_start_us":712,"threads_started":1}
I20260806 16:03:10.367981 13347 heartbeater.cc:347] Connected to a master server at 127.25.31.190:41051
I20260806 16:03:10.368228 13347 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:10.368654 13347 heartbeater.cc:511] Master 127.25.31.190:41051 requested a full tablet report, sending...
I20260806 16:03:10.369989 13154 ts_manager.cc:194] Registered new tserver with Master: 2339206e979f4b778925e288b71ba077 (127.25.31.129:38589)
I20260806 16:03:10.371158 13154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.129@KRBTEST.COM'} at 127.25.31.129:41501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
Entry for principal kudu/127.25.31.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab.
I20260806 16:03:10.380954 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:41051
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:10.488585 13352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:10.488816 13352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:10.488857 13352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:10.492810 13352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:10.492862 13352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:10.492919 13352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:03:10.498059 13352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.130.keytab
--principal=kudu/127.25.31.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:41051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:10.499037 13352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:10.499950 13352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:10.509280 13357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:10.509312 13360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:10.509289 13358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:10.509532 13352 server_base.cc:1034] running on GCE node
I20260806 16:03:10.510018 13352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:10.510650 13352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:10.511870 13352 hybrid_clock.cc:648] HybridClock initialized: now 1786032190511854 us; error 31 us; skew 500 ppm
Aug 06 16:03:10 dist-test-slave-tm1g krb5kdc[13109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032190, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:10.514640 13352 init.cc:377] Logged in from keytab as kudu/127.25.31.130@KRBTEST.COM (short username kudu)
I20260806 16:03:10.515439 13352 webserver.cc:460] Webserver started at http://127.25.31.130:39843/ using document root <none> and password file <none>
I20260806 16:03:10.516036 13352 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:10.516127 13352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:10.516355 13352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:10.518286 13352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "32b450a6006346959bea10819fbc14ce"
format_stamp: "Formatted at 2026-08-06 16:03:10 on dist-test-slave-tm1g"
server_key: "ec84ad4e33d5eed0e33adc4303518544"
server_key_iv: "ffefccd8784dcb71784f4557405553f0"
server_key_version: "encryptionkey@0"
I20260806 16:03:10.518920 13352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "32b450a6006346959bea10819fbc14ce"
format_stamp: "Formatted at 2026-08-06 16:03:10 on dist-test-slave-tm1g"
server_key: "ec84ad4e33d5eed0e33adc4303518544"
server_key_iv: "ffefccd8784dcb71784f4557405553f0"
server_key_version: "encryptionkey@0"
I20260806 16:03:10.522404 13352 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 16:03:10.524667 13367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:10.525799 13352 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:03:10.525940 13352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "32b450a6006346959bea10819fbc14ce"
format_stamp: "Formatted at 2026-08-06 16:03:10 on dist-test-slave-tm1g"
server_key: "ec84ad4e33d5eed0e33adc4303518544"
server_key_iv: "ffefccd8784dcb71784f4557405553f0"
server_key_version: "encryptionkey@0"
I20260806 16:03:10.526047 13352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:10.538789 13352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:10.541936 13352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:10.542129 13352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:10.542740 13352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:10.543521 13352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:10.543592 13352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:10.543665 13352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:10.543709 13352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:10.552469 13352 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:39525
I20260806 16:03:10.552544 13482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:39525 every 8 connection(s)
I20260806 16:03:10.553321 13352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:03:10.556569 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 13352
I20260806 16:03:10.556703 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 16:03:10.556977 25726 external_mini_cluster.cc:1428] Setting key c6ae876419ffc4fac910f669297baf6e
Aug 06 16:03:10 dist-test-slave-tm1g krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032190, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.130@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.31.131@KRBTEST.COM; defaulting to no policy
I20260806 16:03:10.567534 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.555097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:51233 (local address 127.25.31.190:41051)
0806 16:03:10.555236 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:10.555239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:10.555968 (+ 729us) server_negotiation.cc:407] Connection header received
0806 16:03:10.557461 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:10.557464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:10.557535 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:10.557626 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:10.559007 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.559536 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.560678 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.560925 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.561039 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.563823 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:10.563827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:10.563829 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:10.563851 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:10.565353 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:10.565947 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:10.565951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:10.565953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:10.566038 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:10.566398 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:10.566402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:10.566404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:10.566579 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:10.566701 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:10.567237 (+ 536us) server_negotiation.cc:299] Negotiation successful
0806 16:03:10.567369 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:03:10.568389 13485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.555364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:41051 (local address 127.25.31.130:51233)
0806 16:03:10.555809 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:10.555850 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:10.557221 (+ 1371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:10.557815 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:10.557823 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:10.558262 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:10.558856 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.558870 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.559666 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.559670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.560541 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.560550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.561076 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.561080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.561245 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.561864 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:10.561888 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:10.563650 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:10.565482 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:10.565489 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:10.565501 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:10.565800 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:10.566134 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:10.566139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:10.566140 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:10.566275 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:10.566695 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:10.566701 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:10.566980 (+ 279us) client_negotiation.cc:769] Sending connection context
0806 16:03:10.567173 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 16:03:10.567480 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":116,"threads_started":1}
I20260806 16:03:10.569548 13483 heartbeater.cc:347] Connected to a master server at 127.25.31.190:41051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.131@KRBTEST.COM" created.
I20260806 16:03:10.569789 13483 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:10.570261 13483 heartbeater.cc:511] Master 127.25.31.190:41051 requested a full tablet report, sending...
I20260806 16:03:10.571322 13154 ts_manager.cc:194] Registered new tserver with Master: 32b450a6006346959bea10819fbc14ce (127.25.31.130:39525)
I20260806 16:03:10.571905 13154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.130@KRBTEST.COM'} at 127.25.31.130:51233
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
Entry for principal kudu/127.25.31.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab.
I20260806 16:03:10.583483 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:41051
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:32983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:10.691320 13488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:10.691562 13488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:10.691604 13488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:10.695423 13488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:10.695470 13488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:10.695524 13488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:03:10.700469 13488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:32983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.131.keytab
--principal=kudu/127.25.31.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:41051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:10.701359 13488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:10.702288 13488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:10.712234 13493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:10.712235 13494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:10.712244 13496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:10.712404 13488 server_base.cc:1034] running on GCE node
I20260806 16:03:10.712872 13488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:10.713457 13488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:10.714635 13488 hybrid_clock.cc:648] HybridClock initialized: now 1786032190714606 us; error 41 us; skew 500 ppm
Aug 06 16:03:10 dist-test-slave-tm1g krb5kdc[13109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032190, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:10.717347 13488 init.cc:377] Logged in from keytab as kudu/127.25.31.131@KRBTEST.COM (short username kudu)
I20260806 16:03:10.718187 13488 webserver.cc:460] Webserver started at http://127.25.31.131:32773/ using document root <none> and password file <none>
I20260806 16:03:10.718786 13488 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:10.718880 13488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:10.719107 13488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:10.721046 13488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "0a168b77407d404eb9ef9c42657d1e54"
format_stamp: "Formatted at 2026-08-06 16:03:10 on dist-test-slave-tm1g"
server_key: "6a6bf81ea88a87458641505d3893be48"
server_key_iv: "eed676ac783db602531ee1ca3e6bfc57"
server_key_version: "encryptionkey@0"
I20260806 16:03:10.721607 13488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "0a168b77407d404eb9ef9c42657d1e54"
format_stamp: "Formatted at 2026-08-06 16:03:10 on dist-test-slave-tm1g"
server_key: "6a6bf81ea88a87458641505d3893be48"
server_key_iv: "eed676ac783db602531ee1ca3e6bfc57"
server_key_version: "encryptionkey@0"
I20260806 16:03:10.725553 13488 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 16:03:10.728147 13503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:10.729458 13488 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:03:10.729601 13488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "0a168b77407d404eb9ef9c42657d1e54"
format_stamp: "Formatted at 2026-08-06 16:03:10 on dist-test-slave-tm1g"
server_key: "6a6bf81ea88a87458641505d3893be48"
server_key_iv: "eed676ac783db602531ee1ca3e6bfc57"
server_key_version: "encryptionkey@0"
I20260806 16:03:10.729713 13488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:10.750664 13488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:10.753810 13488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:10.754033 13488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:10.754905 13488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:10.756078 13488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:10.756126 13488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:10.756165 13488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:10.756181 13488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:10.764866 13488 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:40091
I20260806 16:03:10.764912 13617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:40091 every 8 connection(s)
I20260806 16:03:10.765750 13488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:03:10.769146 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 13488
I20260806 16:03:10.769266 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 16:03:10.769541 25726 external_mini_cluster.cc:1428] Setting key 4041d23482a0ad6fac6b7a7712b99462
Aug 06 16:03:10 dist-test-slave-tm1g krb5kdc[13109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032190, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.131@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:03:10.779441 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.767597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:34333 (local address 127.25.31.190:41051)
0806 16:03:10.767733 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:10.767737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:10.768532 (+ 795us) server_negotiation.cc:407] Connection header received
0806 16:03:10.770080 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:10.770084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:10.770138 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:10.770230 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:10.771735 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.772285 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.773376 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.773585 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.773662 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.776161 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:10.776164 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:10.776166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:10.776192 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:10.777602 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:10.778055 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:10.778058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:10.778059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:10.778131 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:10.778491 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:10.778494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:10.778495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:10.778664 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:10.778741 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:10.779230 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 16:03:10.779325 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 16:03:10.780172 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.767944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:41051 (local address 127.25.31.131:34333)
0806 16:03:10.768374 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:10.768416 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:10.769815 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:10.770614 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:10.770621 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:10.771053 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:10.771565 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.771585 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.772394 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.772396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.773270 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.773279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.773748 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.773752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.773934 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.774540 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:10.774558 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:10.776010 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:10.777713 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:10.777718 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:10.777728 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:10.777968 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:10.778208 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:10.778210 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:10.778212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:10.778407 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:10.778783 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:10.778788 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:10.779004 (+ 216us) client_negotiation.cc:769] Sending connection context
0806 16:03:10.779167 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 16:03:10.779397 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":121,"threads_started":1}
I20260806 16:03:10.781281 13618 heartbeater.cc:347] Connected to a master server at 127.25.31.190:41051
I20260806 16:03:10.781507 13618 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:10.781955 13618 heartbeater.cc:511] Master 127.25.31.190:41051 requested a full tablet report, sending...
I20260806 16:03:10.783087 13154 ts_manager.cc:194] Registered new tserver with Master: 0a168b77407d404eb9ef9c42657d1e54 (127.25.31.131:40091)
I20260806 16:03:10.783700 13154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.31.131@KRBTEST.COM'} at 127.25.31.131:34333
I20260806 16:03:10.795274 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:03:10.804106 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.797242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34342 (local address 127.25.31.190:41051)
0806 16:03:10.797371 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:10.797374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:10.797568 (+ 194us) server_negotiation.cc:407] Connection header received
0806 16:03:10.797683 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:10.797685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:10.797723 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:10.797790 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:10.798503 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.799002 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.799790 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.799990 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.800061 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.801276 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:10.801278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:10.801280 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:10.801301 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:10.802743 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:10.803093 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:10.803095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:10.803096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:10.803166 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:10.803382 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:10.803384 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:10.803385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:10.803585 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:10.803684 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:10.803872 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 16:03:10.803957 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 16:03:10.806738 13154 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:03:10.809739 13154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:03:10.824060 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.817810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:40091 (local address 127.0.0.1:60968)
0806 16:03:10.818226 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:10.818271 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:10.818423 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:10.819284 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:10.819290 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:10.819316 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:10.819452 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.819459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.820992 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.820998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.822369 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.822380 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.823487 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.823490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.823602 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.823641 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 16:03:10.823761 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 16:03:10.823898 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":22,"thread_start_us":95,"threads_started":1}
I20260806 16:03:10.824070 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.817936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60968 (local address 127.25.31.131:40091)
0806 16:03:10.818736 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:10.818749 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:10.818771 (+ 22us) server_negotiation.cc:407] Connection header received
0806 16:03:10.818867 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:10.818875 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:10.819098 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:10.819612 (+ 514us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:10.819751 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.820869 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.822504 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.823352 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.823467 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.823611 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:10.823812 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 16:03:10.823924 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":142,"threads_started":1}
I20260806 16:03:10.824440 13630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.817901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:39525 (local address 127.0.0.1:37960)
0806 16:03:10.818947 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:10.818960 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:10.819118 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:10.819778 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:10.819781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:10.819798 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:10.819907 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.819913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.821374 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.821378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.822660 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.822670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.823966 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.823970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.824070 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.824102 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 16:03:10.824176 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 16:03:10.824253 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"mutex_wait_us":115,"thread_start_us":114,"threads_started":1}
I20260806 16:03:10.824601 13632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.818350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37960 (local address 127.25.31.130:39525)
0806 16:03:10.818788 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:10.818796 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:10.819044 (+ 248us) server_negotiation.cc:407] Connection header received
0806 16:03:10.819280 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:10.819285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:10.819472 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:10.819571 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:10.820036 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.821216 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.822800 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.823625 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.823705 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.823867 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:10.824354 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 16:03:10.824452 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":128,"threads_started":1}
I20260806 16:03:10.824810 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.819457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38226 (local address 127.25.31.129:38589)
0806 16:03:10.819831 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:10.819842 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:10.819862 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:03:10.819952 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:10.819960 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:10.820159 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:10.820313 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:10.820562 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.821564 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.823062 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.823741 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.824370 (+ 629us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.824519 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:10.824572 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 16:03:10.824670 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":138,"threads_started":1}
I20260806 16:03:10.824265 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.818966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:38589 (local address 127.0.0.1:38226)
0806 16:03:10.819197 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:10.819212 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:10.819285 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:10.820309 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:10.820312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:10.820341 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:10.820428 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.820434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.821825 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.821827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.822946 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.822955 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.823878 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.823881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.823974 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.823995 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 16:03:10.824044 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 16:03:10.824115 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":56,"thread_start_us":105,"threads_started":1}
I20260806 16:03:10.827700 13280 tablet_service.cc:1467] Processing CreateTablet for tablet 1b6a56a032584596a66b0bbbf2602d25 (DEFAULT_TABLE table=test-table [id=06d8bfa8b96b42279ba8c49726efa8d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:10.827540 13551 tablet_service.cc:1467] Processing CreateTablet for tablet 1b6a56a032584596a66b0bbbf2602d25 (DEFAULT_TABLE table=test-table [id=06d8bfa8b96b42279ba8c49726efa8d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:10.827540 13416 tablet_service.cc:1467] Processing CreateTablet for tablet 1b6a56a032584596a66b0bbbf2602d25 (DEFAULT_TABLE table=test-table [id=06d8bfa8b96b42279ba8c49726efa8d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:10.828619 13280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b6a56a032584596a66b0bbbf2602d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:10.828632 13416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b6a56a032584596a66b0bbbf2602d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:10.828632 13551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b6a56a032584596a66b0bbbf2602d25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:10.835214 13635 tablet_bootstrap.cc:492] T 1b6a56a032584596a66b0bbbf2602d25 P 2339206e979f4b778925e288b71ba077: Bootstrap starting.
I20260806 16:03:10.836329 13637 tablet_bootstrap.cc:492] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce: Bootstrap starting.
I20260806 16:03:10.836790 13636 tablet_bootstrap.cc:492] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54: Bootstrap starting.
I20260806 16:03:10.837944 13635 tablet_bootstrap.cc:654] T 1b6a56a032584596a66b0bbbf2602d25 P 2339206e979f4b778925e288b71ba077: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:10.838052 13637 tablet_bootstrap.cc:654] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:10.838804 13636 tablet_bootstrap.cc:654] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:10.839202 13637 log.cc:824] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:10.839309 13635 log.cc:824] T 1b6a56a032584596a66b0bbbf2602d25 P 2339206e979f4b778925e288b71ba077: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:10.840030 13636 log.cc:824] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:10.841218 13637 tablet_bootstrap.cc:492] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce: No bootstrap required, opened a new log
I20260806 16:03:10.841413 13637 ts_tablet_manager.cc:1397] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 16:03:10.841710 13636 tablet_bootstrap.cc:492] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54: No bootstrap required, opened a new log
I20260806 16:03:10.841892 13636 ts_tablet_manager.cc:1397] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 16:03:10.843465 13635 tablet_bootstrap.cc:492] T 1b6a56a032584596a66b0bbbf2602d25 P 2339206e979f4b778925e288b71ba077: No bootstrap required, opened a new log
I20260806 16:03:10.843679 13635 ts_tablet_manager.cc:1397] T 1b6a56a032584596a66b0bbbf2602d25 P 2339206e979f4b778925e288b71ba077: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 16:03:10.844781 13636 raft_consensus.cc:348] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a168b77407d404eb9ef9c42657d1e54" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40091 } } peers { permanent_uuid: "32b450a6006346959bea10819fbc14ce" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39525 } } peers { permanent_uuid: "2339206e979f4b778925e288b71ba077" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 38589 } }
I20260806 16:03:10.845012 13636 raft_consensus.cc:374] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:10.845067 13636 raft_consensus.cc:729] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a168b77407d404eb9ef9c42657d1e54, State: Initialized, Role: FOLLOWER
I20260806 16:03:10.845140 13637 raft_consensus.cc:348] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a168b77407d404eb9ef9c42657d1e54" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40091 } } peers { permanent_uuid: "32b450a6006346959bea10819fbc14ce" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39525 } } peers { permanent_uuid: "2339206e979f4b778925e288b71ba077" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 38589 } }
I20260806 16:03:10.845412 13637 raft_consensus.cc:374] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:10.845464 13637 raft_consensus.cc:729] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32b450a6006346959bea10819fbc14ce, State: Initialized, Role: FOLLOWER
I20260806 16:03:10.845665 13636 consensus_queue.cc:260] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a168b77407d404eb9ef9c42657d1e54" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40091 } } peers { permanent_uuid: "32b450a6006346959bea10819fbc14ce" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39525 } } peers { permanent_uuid: "2339206e979f4b778925e288b71ba077" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 38589 } }
I20260806 16:03:10.845974 13637 consensus_queue.cc:260] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a168b77407d404eb9ef9c42657d1e54" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40091 } } peers { permanent_uuid: "32b450a6006346959bea10819fbc14ce" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39525 } } peers { permanent_uuid: "2339206e979f4b778925e288b71ba077" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 38589 } }
I20260806 16:03:10.847069 13637 ts_tablet_manager.cc:1428] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 16:03:10.847077 13636 ts_tablet_manager.cc:1428] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 16:03:10.847430 13483 heartbeater.cc:503] Master 127.25.31.190:41051 was elected leader, sending a full tablet report...
I20260806 16:03:10.847961 13635 raft_consensus.cc:348] T 1b6a56a032584596a66b0bbbf2602d25 P 2339206e979f4b778925e288b71ba077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a168b77407d404eb9ef9c42657d1e54" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40091 } } peers { permanent_uuid: "32b450a6006346959bea10819fbc14ce" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39525 } } peers { permanent_uuid: "2339206e979f4b778925e288b71ba077" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 38589 } }
I20260806 16:03:10.848170 13635 raft_consensus.cc:374] T 1b6a56a032584596a66b0bbbf2602d25 P 2339206e979f4b778925e288b71ba077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:10.848251 13635 raft_consensus.cc:729] T 1b6a56a032584596a66b0bbbf2602d25 P 2339206e979f4b778925e288b71ba077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2339206e979f4b778925e288b71ba077, State: Initialized, Role: FOLLOWER
I20260806 16:03:10.848343 13618 heartbeater.cc:503] Master 127.25.31.190:41051 was elected leader, sending a full tablet report...
I20260806 16:03:10.848637 13635 consensus_queue.cc:260] T 1b6a56a032584596a66b0bbbf2602d25 P 2339206e979f4b778925e288b71ba077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a168b77407d404eb9ef9c42657d1e54" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40091 } } peers { permanent_uuid: "32b450a6006346959bea10819fbc14ce" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39525 } } peers { permanent_uuid: "2339206e979f4b778925e288b71ba077" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 38589 } }
I20260806 16:03:10.849282 13347 heartbeater.cc:503] Master 127.25.31.190:41051 was elected leader, sending a full tablet report...
I20260806 16:03:10.849531 13635 ts_tablet_manager.cc:1428] T 1b6a56a032584596a66b0bbbf2602d25 P 2339206e979f4b778925e288b71ba077: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260806 16:03:10.852707 13348 tablet.cc:2369] T 1b6a56a032584596a66b0bbbf2602d25 P 2339206e979f4b778925e288b71ba077: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:03:10.863677 13641 raft_consensus.cc:482] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:03:10.863835 13641 raft_consensus.cc:504] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a168b77407d404eb9ef9c42657d1e54" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40091 } } peers { permanent_uuid: "32b450a6006346959bea10819fbc14ce" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39525 } } peers { permanent_uuid: "2339206e979f4b778925e288b71ba077" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 38589 } }
I20260806 16:03:10.864841 13641 leader_election.cc:288] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32b450a6006346959bea10819fbc14ce (127.25.31.130:39525), 2339206e979f4b778925e288b71ba077 (127.25.31.129:38589)
I20260806 16:03:10.869011 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.865024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:39525 (local address 127.25.31.131:36561)
0806 16:03:10.865212 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:10.865228 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:10.865335 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:10.865788 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:10.865791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:10.865808 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:10.865940 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.865946 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.867123 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.867126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.867948 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.867957 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.868593 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.868597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.868700 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.868726 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 16:03:10.868811 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 16:03:10.868884 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 16:03:10.869011 13632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.865173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:36561 (local address 127.25.31.130:39525)
0806 16:03:10.865293 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:10.865311 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:10.865327 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:03:10.865483 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:10.865486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:10.865545 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:10.865614 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:10.866058 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.866988 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.868070 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.868473 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.868603 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.868657 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:10.868836 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 16:03:10.868906 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 16:03:10.869205 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.865528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:59459 (local address 127.25.31.129:38589)
0806 16:03:10.865658 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:10.865663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:10.865762 (+ 99us) server_negotiation.cc:407] Connection header received
0806 16:03:10.865935 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:10.865943 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:10.865983 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:10.866086 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:10.866442 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.867146 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.868152 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.868678 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.868803 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.868848 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:10.869034 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 16:03:10.869102 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 16:03:10.869231 13644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.865277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:38589 (local address 127.25.31.131:59459)
0806 16:03:10.865668 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:10.865681 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:10.865836 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:10.866103 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:10.866105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:10.866121 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:10.866213 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.866217 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.867257 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.867262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.868029 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:10.868037 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.868797 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.868801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:10.868901 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.868929 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 16:03:10.869019 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 16:03:10.869087 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"spinlock_wait_cycles":768,"thread_start_us":103,"threads_started":1}
I20260806 16:03:10.869866 13436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6a56a032584596a66b0bbbf2602d25" candidate_uuid: "0a168b77407d404eb9ef9c42657d1e54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32b450a6006346959bea10819fbc14ce" is_pre_election: true
I20260806 16:03:10.869943 13300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6a56a032584596a66b0bbbf2602d25" candidate_uuid: "0a168b77407d404eb9ef9c42657d1e54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2339206e979f4b778925e288b71ba077" is_pre_election: true
I20260806 16:03:10.870128 13436 raft_consensus.cc:2454] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a168b77407d404eb9ef9c42657d1e54 in term 0.
I20260806 16:03:10.870254 13300 raft_consensus.cc:2454] T 1b6a56a032584596a66b0bbbf2602d25 P 2339206e979f4b778925e288b71ba077 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a168b77407d404eb9ef9c42657d1e54 in term 0.
I20260806 16:03:10.870606 13504 leader_election.cc:302] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a168b77407d404eb9ef9c42657d1e54, 32b450a6006346959bea10819fbc14ce; no voters:
I20260806 16:03:10.870860 13641 raft_consensus.cc:2781] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:03:10.870954 13641 raft_consensus.cc:482] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:03:10.871006 13641 raft_consensus.cc:3037] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:10.872011 13641 raft_consensus.cc:504] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a168b77407d404eb9ef9c42657d1e54" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40091 } } peers { permanent_uuid: "32b450a6006346959bea10819fbc14ce" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39525 } } peers { permanent_uuid: "2339206e979f4b778925e288b71ba077" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 38589 } }
I20260806 16:03:10.872318 13641 leader_election.cc:288] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [CANDIDATE]: Term 1 election: Requested vote from peers 32b450a6006346959bea10819fbc14ce (127.25.31.130:39525), 2339206e979f4b778925e288b71ba077 (127.25.31.129:38589)
I20260806 16:03:10.872624 13436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6a56a032584596a66b0bbbf2602d25" candidate_uuid: "0a168b77407d404eb9ef9c42657d1e54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32b450a6006346959bea10819fbc14ce"
I20260806 16:03:10.872694 13300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6a56a032584596a66b0bbbf2602d25" candidate_uuid: "0a168b77407d404eb9ef9c42657d1e54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2339206e979f4b778925e288b71ba077"
I20260806 16:03:10.872749 13436 raft_consensus.cc:3037] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:10.872852 13300 raft_consensus.cc:3037] T 1b6a56a032584596a66b0bbbf2602d25 P 2339206e979f4b778925e288b71ba077 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:10.873785 13436 raft_consensus.cc:2454] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a168b77407d404eb9ef9c42657d1e54 in term 1.
I20260806 16:03:10.873883 13300 raft_consensus.cc:2454] T 1b6a56a032584596a66b0bbbf2602d25 P 2339206e979f4b778925e288b71ba077 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a168b77407d404eb9ef9c42657d1e54 in term 1.
I20260806 16:03:10.874125 13504 leader_election.cc:302] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a168b77407d404eb9ef9c42657d1e54, 32b450a6006346959bea10819fbc14ce; no voters:
I20260806 16:03:10.874375 13641 raft_consensus.cc:2781] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:10.874591 13641 raft_consensus.cc:686] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [term 1 LEADER]: Becoming Leader. State: Replica: 0a168b77407d404eb9ef9c42657d1e54, State: Running, Role: LEADER
I20260806 16:03:10.874898 13641 consensus_queue.cc:237] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a168b77407d404eb9ef9c42657d1e54" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40091 } } peers { permanent_uuid: "32b450a6006346959bea10819fbc14ce" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39525 } } peers { permanent_uuid: "2339206e979f4b778925e288b71ba077" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 38589 } }
I20260806 16:03:10.877827 13153 catalog_manager.cc:5495] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a168b77407d404eb9ef9c42657d1e54 (127.25.31.131). New cstate: current_term: 1 leader_uuid: "0a168b77407d404eb9ef9c42657d1e54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a168b77407d404eb9ef9c42657d1e54" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 40091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32b450a6006346959bea10819fbc14ce" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2339206e979f4b778925e288b71ba077" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 38589 } health_report { overall_health: UNKNOWN } } }
I20260806 16:03:10.901252 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.898053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60970 (local address 127.25.31.131:40091)
0806 16:03:10.898179 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:10.898183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:10.898196 (+ 13us) server_negotiation.cc:407] Connection header received
0806 16:03:10.898337 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:10.898347 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:10.898427 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:10.898509 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:10.898721 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.899198 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.899986 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:10.900187 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:10.900260 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:10.900503 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:10.900871 (+ 368us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:10.900957 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:10.901014 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 16:03:10.901061 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 16:03:10.906780 13436 raft_consensus.cc:1261] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce [term 1 FOLLOWER]: Refusing update from remote peer 0a168b77407d404eb9ef9c42657d1e54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:10.906804 13300 raft_consensus.cc:1261] T 1b6a56a032584596a66b0bbbf2602d25 P 2339206e979f4b778925e288b71ba077 [term 1 FOLLOWER]: Refusing update from remote peer 0a168b77407d404eb9ef9c42657d1e54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:10.907473 13641 consensus_queue.cc:1035] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32b450a6006346959bea10819fbc14ce" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 39525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:03:10.907692 13645 consensus_queue.cc:1035] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2339206e979f4b778925e288b71ba077" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 38589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:03:10.915135 13650 mvcc.cc:205] Tried to move back new op lower bound from 7315587853947154432 to 7315587853825961984. Current Snapshot: MvccSnapshot[applied={T|T < 7315587853947154432}]
I20260806 16:03:10.917203 13649 mvcc.cc:205] Tried to move back new op lower bound from 7315587853947154432 to 7315587853825961984. Current Snapshot: MvccSnapshot[applied={T|T < 7315587853947154432}]
I20260806 16:03:10.922569 13153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:03:10.922734 13153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:03:10.924985 13153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65948732b33d420c9bcc4c2be4270dbd: Sending DeleteTablet for 3 replicas of tablet 1b6a56a032584596a66b0bbbf2602d25
I20260806 16:03:10.925743 13551 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b6a56a032584596a66b0bbbf2602d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:10 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:60968
I20260806 16:03:10.925853 13416 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b6a56a032584596a66b0bbbf2602d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:10 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:37960
I20260806 16:03:10.925928 13280 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b6a56a032584596a66b0bbbf2602d25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:10 UTC) from {username='kudu', principal='kudu/127.25.31.190@KRBTEST.COM'} at 127.0.0.1:38226
I20260806 16:03:10.926335 13659 tablet_replica.cc:330] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce: stopping tablet replica
I20260806 16:03:10.926518 13658 tablet_replica.cc:330] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54: stopping tablet replica
I20260806 16:03:10.926630 13659 raft_consensus.cc:2229] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:03:10.926890 13658 raft_consensus.cc:2229] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:03:10.926921 13659 raft_consensus.cc:2258] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:03:10.926982 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 13217
I20260806 16:03:10.927342 13658 raft_consensus.cc:2258] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:03:10.929574 13658 ts_tablet_manager.cc:1905] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:03:10.930819 13659 ts_tablet_manager.cc:1905] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:03:10.932510 13658 ts_tablet_manager.cc:1918] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:03:10.932693 13658 log.cc:1196] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/1b6a56a032584596a66b0bbbf2602d25
I20260806 16:03:10.933035 13658 ts_tablet_manager.cc:1939] T 1b6a56a032584596a66b0bbbf2602d25 P 0a168b77407d404eb9ef9c42657d1e54: Deleting consensus metadata
I20260806 16:03:10.933218 13659 ts_tablet_manager.cc:1918] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:03:10.933300 13659 log.cc:1196] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786031939584994-25726-0/minicluster-data/ts-1/wal/wals/1b6a56a032584596a66b0bbbf2602d25
I20260806 16:03:10.933633 13659 ts_tablet_manager.cc:1939] T 1b6a56a032584596a66b0bbbf2602d25 P 32b450a6006346959bea10819fbc14ce: Deleting consensus metadata
I20260806 16:03:10.934139 13140 catalog_manager.cc:4841] TS 0a168b77407d404eb9ef9c42657d1e54 (127.25.31.131:40091): tablet 1b6a56a032584596a66b0bbbf2602d25 (table test-table [id=06d8bfa8b96b42279ba8c49726efa8d7]) successfully deleted
I20260806 16:03:10.934778 13138 catalog_manager.cc:4841] TS 32b450a6006346959bea10819fbc14ce (127.25.31.130:39525): tablet 1b6a56a032584596a66b0bbbf2602d25 (table test-table [id=06d8bfa8b96b42279ba8c49726efa8d7]) successfully deleted
W20260806 16:03:10.940233 13138 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:38589 (error 108)
I20260806 16:03:10.941056 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 13352
I20260806 16:03:10.941254 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:10.940962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:38589 (local address 127.0.0.1:38236)
0806 16:03:10.941104 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:10.941173 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:38589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 16:03:10.942214 13138 catalog_manager.cc:4584] TS 2339206e979f4b778925e288b71ba077 (127.25.31.129:38589): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1b6a56a032584596a66b0bbbf2602d25: Network error: Client connection negotiation failed: client connection to 127.25.31.129:38589: connect: Connection refused (error 111)
I20260806 16:03:10.948556 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 13488
I20260806 16:03:10.954118 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 13122
2026-08-06T16:03:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2234 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T16:03:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:10Z Disabled control of system clock
I20260806 16:03:10.985895 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:35641
--webserver_interface=127.25.31.190
--webserver_port=0
--builtin_ntp_servers=127.25.31.148:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:35641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:03:11.093719 13665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:11.093955 13665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:11.093990 13665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:11.097875 13665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:11.097927 13665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:11.097944 13665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:11.097959 13665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:11.097975 13665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:11.102635 13665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:35641
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:35641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:11.103570 13665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:11.104831 13665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:11.113147 13673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:11.113157 13670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:11.113168 13671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:11.113210 13665 server_base.cc:1034] running on GCE node
I20260806 16:03:11.113921 13665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:11.115020 13665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:11.116211 13665 hybrid_clock.cc:648] HybridClock initialized: now 1786032191116189 us; error 42 us; skew 500 ppm
I20260806 16:03:11.117995 13665 webserver.cc:460] Webserver started at http://127.25.31.190:43435/ using document root <none> and password file <none>
I20260806 16:03:11.118659 13665 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:11.118752 13665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:11.118980 13665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:11.120980 13665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "8c9940c26bd5479ca2383c4979d75e7b"
format_stamp: "Formatted at 2026-08-06 16:03:11 on dist-test-slave-tm1g"
server_key: "a79a1f34cd4f35c2af14cbc832cb3131"
server_key_iv: "21bf4a5e226aa1287086fca61f0fc7ee"
server_key_version: "encryptionkey@0"
I20260806 16:03:11.121541 13665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "8c9940c26bd5479ca2383c4979d75e7b"
format_stamp: "Formatted at 2026-08-06 16:03:11 on dist-test-slave-tm1g"
server_key: "a79a1f34cd4f35c2af14cbc832cb3131"
server_key_iv: "21bf4a5e226aa1287086fca61f0fc7ee"
server_key_version: "encryptionkey@0"
I20260806 16:03:11.125015 13665 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260806 16:03:11.127323 13679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:11.128451 13665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:03:11.128584 13665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "8c9940c26bd5479ca2383c4979d75e7b"
format_stamp: "Formatted at 2026-08-06 16:03:11 on dist-test-slave-tm1g"
server_key: "a79a1f34cd4f35c2af14cbc832cb3131"
server_key_iv: "21bf4a5e226aa1287086fca61f0fc7ee"
server_key_version: "encryptionkey@0"
I20260806 16:03:11.128691 13665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:11.144730 13665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:11.145354 13665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:11.145545 13665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:11.152192 13665 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:35641
I20260806 16:03:11.152204 13731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:35641 every 8 connection(s)
I20260806 16:03:11.153075 13665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:11.156085 13733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:11.161237 13733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b: Bootstrap starting.
I20260806 16:03:11.161478 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 13665
I20260806 16:03:11.161608 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:11.161918 25726 external_mini_cluster.cc:1428] Setting key 8db0351ee7651fe8853ee1e218e11b1b
I20260806 16:03:11.163954 13733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:11.164670 13733 log.cc:824] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:11.166712 13733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b: No bootstrap required, opened a new log
I20260806 16:03:11.171012 13733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9940c26bd5479ca2383c4979d75e7b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35641 } }
I20260806 16:03:11.171216 13733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:11.171263 13736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.163264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.25.31.190:35641)
0806 16:03:11.163871 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:11.163885 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:11.163917 (+ 32us) server_negotiation.cc:407] Connection header received
0806 16:03:11.164567 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:11.164583 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:11.165032 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:11.165404 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:11.165751 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.166790 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.168093 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.168561 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.168676 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.169537 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:11.169541 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:11.169553 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:11.169925 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:11.169983 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:11.170001 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:11.170075 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:11.170265 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 16:03:11.170536 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":117,"threads_started":1}
I20260806 16:03:11.171283 13733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9940c26bd5479ca2383c4979d75e7b, State: Initialized, Role: FOLLOWER
I20260806 16:03:11.171751 13733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9940c26bd5479ca2383c4979d75e7b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35641 } }
I20260806 16:03:11.171875 13733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:11.171964 13733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:11.172096 13733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:11.173182 13733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9940c26bd5479ca2383c4979d75e7b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35641 } }
I20260806 16:03:11.173529 13733 leader_election.cc:302] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c9940c26bd5479ca2383c4979d75e7b; no voters:
I20260806 16:03:11.173849 13733 leader_election.cc:288] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:11.173998 13738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:11.174230 13738 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [term 1 LEADER]: Becoming Leader. State: Replica: 8c9940c26bd5479ca2383c4979d75e7b, State: Running, Role: LEADER
I20260806 16:03:11.174575 13738 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9940c26bd5479ca2383c4979d75e7b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35641 } }
I20260806 16:03:11.175094 13733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:11.176139 13740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c9940c26bd5479ca2383c4979d75e7b. Latest consensus state: current_term: 1 leader_uuid: "8c9940c26bd5479ca2383c4979d75e7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9940c26bd5479ca2383c4979d75e7b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35641 } } }
I20260806 16:03:11.176267 13740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:11.176553 13739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c9940c26bd5479ca2383c4979d75e7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c9940c26bd5479ca2383c4979d75e7b" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35641 } } }
I20260806 16:03:11.176651 13739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:11.176754 13747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:11.179562 13747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:11.184506 13747 catalog_manager.cc:1347] Generated new cluster ID: 12adb251652b4bea865908cee355ba32
I20260806 16:03:11.184571 13747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:11.201949 13747 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:11.202916 13747 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:11.207196 13747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b: Generated new TSK 0
I20260806 16:03:11.207836 13747 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:11.216840 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.129:0
--local_ip_for_outbound_sockets=127.25.31.129
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35641
--builtin_ntp_servers=127.25.31.148:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 16:03:11.325861 13757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:11.326092 13757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:11.326129 13757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:11.330080 13757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:11.330128 13757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:11.330179 13757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.129
I20260806 16:03:11.335001 13757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.31.129
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.129
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:11.335973 13757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:11.337284 13757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:11.346534 13765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:11.346539 13762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:11.346544 13763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:11.346819 13757 server_base.cc:1034] running on GCE node
I20260806 16:03:11.347334 13757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:11.347915 13757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:11.349076 13757 hybrid_clock.cc:648] HybridClock initialized: now 1786032191349057 us; error 30 us; skew 500 ppm
I20260806 16:03:11.350898 13757 webserver.cc:460] Webserver started at http://127.25.31.129:34669/ using document root <none> and password file <none>
I20260806 16:03:11.351450 13757 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:11.351511 13757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:11.351744 13757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:11.353668 13757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/data/instance:
uuid: "45bbaf0a2630437e80886cafe93cdd1f"
format_stamp: "Formatted at 2026-08-06 16:03:11 on dist-test-slave-tm1g"
server_key: "8242fa73a248a0141a3ec4e5be15af6e"
server_key_iv: "d6be253061f518cef8cb4b82afef4713"
server_key_version: "encryptionkey@0"
I20260806 16:03:11.354214 13757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance:
uuid: "45bbaf0a2630437e80886cafe93cdd1f"
format_stamp: "Formatted at 2026-08-06 16:03:11 on dist-test-slave-tm1g"
server_key: "8242fa73a248a0141a3ec4e5be15af6e"
server_key_iv: "d6be253061f518cef8cb4b82afef4713"
server_key_version: "encryptionkey@0"
I20260806 16:03:11.357774 13757 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:03:11.360123 13771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:11.361251 13757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:03:11.361377 13757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/wal
uuid: "45bbaf0a2630437e80886cafe93cdd1f"
format_stamp: "Formatted at 2026-08-06 16:03:11 on dist-test-slave-tm1g"
server_key: "8242fa73a248a0141a3ec4e5be15af6e"
server_key_iv: "d6be253061f518cef8cb4b82afef4713"
server_key_version: "encryptionkey@0"
I20260806 16:03:11.361474 13757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:11.395249 13757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:11.395951 13757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:11.396155 13757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:11.396751 13757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:11.397673 13757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:11.397744 13757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:11.397816 13757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:11.397867 13757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:11.406716 13757 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.129:37061
I20260806 16:03:11.406746 13888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.129:37061 every 8 connection(s)
I20260806 16:03:11.407577 13757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/data/info.pb
I20260806 16:03:11.413441 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 13757
I20260806 16:03:11.413566 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-0/wal/instance
I20260806 16:03:11.413902 25726 external_mini_cluster.cc:1428] Setting key a868d05988628a3e3014eecf943f8544
I20260806 16:03:11.415781 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.130:0
--local_ip_for_outbound_sockets=127.25.31.130
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35641
--builtin_ntp_servers=127.25.31.148:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:03:11.417403 13736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.409340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.129:54381 (local address 127.25.31.190:35641)
0806 16:03:11.409483 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:11.409486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:11.410529 (+ 1043us) server_negotiation.cc:407] Connection header received
0806 16:03:11.411922 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:11.411926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:11.411990 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:11.412092 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:11.412951 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.413716 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.415190 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.415519 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.415583 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.416848 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:11.416855 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:11.416862 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:11.416880 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:11.416906 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:11.416920 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:11.417001 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:11.417142 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 16:03:11.417217 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 16:03:11.418120 13891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.409704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:35641 (local address 127.25.31.129:54381)
0806 16:03:11.410369 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:11.410422 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:11.411704 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:11.412259 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:11.412265 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:11.412352 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:11.412800 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.412812 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.413846 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.413850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.415045 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.415057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.415733 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.415736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.415953 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.416590 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:03:11.416592 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:11.416629 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:03:11.416688 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:11.417007 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:11.417015 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:11.417019 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 16:03:11.417112 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 16:03:11.417320 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":115,"threads_started":1}
I20260806 16:03:11.419361 13889 heartbeater.cc:347] Connected to a master server at 127.25.31.190:35641
I20260806 16:03:11.419597 13889 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:11.420077 13889 heartbeater.cc:511] Master 127.25.31.190:35641 requested a full tablet report, sending...
I20260806 16:03:11.421473 13696 ts_manager.cc:194] Registered new tserver with Master: 45bbaf0a2630437e80886cafe93cdd1f (127.25.31.129:37061)
I20260806 16:03:11.422663 13696 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.31.129:54381
W20260806 16:03:11.526487 13892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:11.526741 13892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:11.526804 13892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:11.530773 13892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:11.530843 13892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:11.530922 13892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.130
I20260806 16:03:11.535676 13892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.31.130
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.130
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:11.536659 13892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:11.537994 13892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:11.547303 13898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:11.547286 13900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:11.547277 13897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:11.547642 13892 server_base.cc:1034] running on GCE node
I20260806 16:03:11.547972 13892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:11.548552 13892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:11.549731 13892 hybrid_clock.cc:648] HybridClock initialized: now 1786032191549699 us; error 41 us; skew 500 ppm
I20260806 16:03:11.551640 13892 webserver.cc:460] Webserver started at http://127.25.31.130:37391/ using document root <none> and password file <none>
I20260806 16:03:11.552184 13892 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:11.552246 13892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:11.552488 13892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:11.554503 13892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/data/instance:
uuid: "ffe3eb93a5f64765a2af79d3c52b00fb"
format_stamp: "Formatted at 2026-08-06 16:03:11 on dist-test-slave-tm1g"
server_key: "29f41fb0bb90bc6a5443494fddb0fe7a"
server_key_iv: "d959ecec80223518d5363765413327a8"
server_key_version: "encryptionkey@0"
I20260806 16:03:11.555048 13892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance:
uuid: "ffe3eb93a5f64765a2af79d3c52b00fb"
format_stamp: "Formatted at 2026-08-06 16:03:11 on dist-test-slave-tm1g"
server_key: "29f41fb0bb90bc6a5443494fddb0fe7a"
server_key_iv: "d959ecec80223518d5363765413327a8"
server_key_version: "encryptionkey@0"
I20260806 16:03:11.558576 13892 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 16:03:11.560866 13906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:11.562036 13892 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:03:11.562126 13892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/wal
uuid: "ffe3eb93a5f64765a2af79d3c52b00fb"
format_stamp: "Formatted at 2026-08-06 16:03:11 on dist-test-slave-tm1g"
server_key: "29f41fb0bb90bc6a5443494fddb0fe7a"
server_key_iv: "d959ecec80223518d5363765413327a8"
server_key_version: "encryptionkey@0"
I20260806 16:03:11.562245 13892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:11.572216 13892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:11.572801 13892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:11.573025 13892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:11.573563 13892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:11.574487 13892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:11.574553 13892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:11.574648 13892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:11.574728 13892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:11.583477 13892 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.130:42359
I20260806 16:03:11.583509 14020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.130:42359 every 8 connection(s)
I20260806 16:03:11.584295 13892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/data/info.pb
I20260806 16:03:11.592730 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 13892
I20260806 16:03:11.592892 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-1/wal/instance
I20260806 16:03:11.593163 25726 external_mini_cluster.cc:1428] Setting key 03de359a91ba96407e696365f79ad450
I20260806 16:03:11.593302 13736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.585975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.130:47359 (local address 127.25.31.190:35641)
0806 16:03:11.586131 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:11.586135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:11.587118 (+ 983us) server_negotiation.cc:407] Connection header received
0806 16:03:11.588510 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:11.588514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:11.588573 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:11.588681 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:11.589582 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.590149 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.591320 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.591550 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.591666 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.592748 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:11.592751 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:11.592753 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:11.592764 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:11.592784 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:11.592792 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:11.592884 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:11.593095 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 16:03:11.593158 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 16:03:11.594190 14023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.586349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:35641 (local address 127.25.31.130:47359)
0806 16:03:11.586972 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:11.587011 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:11.588293 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:11.588854 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:11.588860 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:11.588952 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:11.589419 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.589431 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.590294 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.590314 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.591179 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.591188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.591704 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.591708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.591873 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.592429 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:03:11.592431 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:11.592497 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:03:11.592581 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:11.592921 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:11.592932 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:11.592940 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 16:03:11.593079 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 16:03:11.593345 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":110,"threads_started":1}
I20260806 16:03:11.595237 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.31.131:0
--local_ip_for_outbound_sockets=127.25.31.131
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35641
--builtin_ntp_servers=127.25.31.148:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 16:03:11.595345 14021 heartbeater.cc:347] Connected to a master server at 127.25.31.190:35641
I20260806 16:03:11.595564 14021 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:11.596141 14021 heartbeater.cc:511] Master 127.25.31.190:35641 requested a full tablet report, sending...
I20260806 16:03:11.597174 13696 ts_manager.cc:194] Registered new tserver with Master: ffe3eb93a5f64765a2af79d3c52b00fb (127.25.31.130:42359)
I20260806 16:03:11.597740 13696 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.31.130:47359
W20260806 16:03:11.705204 14024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:11.705499 14024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:11.705577 14024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:11.709750 14024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:11.709821 14024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:11.709903 14024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.31.131
I20260806 16:03:11.714653 14024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.31.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.31.131
--webserver_port=0
--tserver_master_addrs=127.25.31.190:35641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.31.131
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:11.715638 14024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:11.717027 14024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:11.726531 14030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:11.726452 14031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:11.726452 14033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:11.726569 14024 server_base.cc:1034] running on GCE node
I20260806 16:03:11.727142 14024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:11.727723 14024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:11.728936 14024 hybrid_clock.cc:648] HybridClock initialized: now 1786032191728909 us; error 38 us; skew 500 ppm
I20260806 16:03:11.730790 14024 webserver.cc:460] Webserver started at http://127.25.31.131:42909/ using document root <none> and password file <none>
I20260806 16:03:11.731349 14024 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:11.731413 14024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:11.731619 14024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:11.733619 14024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/data/instance:
uuid: "a895ff7c039e4cf3ab331a18d209e495"
format_stamp: "Formatted at 2026-08-06 16:03:11 on dist-test-slave-tm1g"
server_key: "3bd9436350107d5ec8b59b74488632cf"
server_key_iv: "d9dc87bc1df571ff7463b92701623042"
server_key_version: "encryptionkey@0"
I20260806 16:03:11.734170 14024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance:
uuid: "a895ff7c039e4cf3ab331a18d209e495"
format_stamp: "Formatted at 2026-08-06 16:03:11 on dist-test-slave-tm1g"
server_key: "3bd9436350107d5ec8b59b74488632cf"
server_key_iv: "d9dc87bc1df571ff7463b92701623042"
server_key_version: "encryptionkey@0"
I20260806 16:03:11.737725 14024 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 16:03:11.740751 14039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:11.742370 14024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:03:11.742504 14024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/wal
uuid: "a895ff7c039e4cf3ab331a18d209e495"
format_stamp: "Formatted at 2026-08-06 16:03:11 on dist-test-slave-tm1g"
server_key: "3bd9436350107d5ec8b59b74488632cf"
server_key_iv: "d9dc87bc1df571ff7463b92701623042"
server_key_version: "encryptionkey@0"
I20260806 16:03:11.742628 14024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:11.769016 14024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:11.769621 14024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:11.769800 14024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:11.770383 14024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 16:03:11.771272 14024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 16:03:11.771320 14024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:11.771358 14024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 16:03:11.771374 14024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:11.780485 14024 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.131:35089
I20260806 16:03:11.780525 14154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.131:35089 every 8 connection(s)
I20260806 16:03:11.781371 14024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/data/info.pb
I20260806 16:03:11.782110 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 14024
I20260806 16:03:11.782203 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/wal/instance
I20260806 16:03:11.782553 25726 external_mini_cluster.cc:1428] Setting key 11f369497a3a5774e29fb15e62ac18e5
I20260806 16:03:11.790448 13736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.783297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:49527 (local address 127.25.31.190:35641)
0806 16:03:11.783410 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:11.783415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:11.784686 (+ 1271us) server_negotiation.cc:407] Connection header received
0806 16:03:11.786085 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:11.786088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:11.786132 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:11.786199 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:11.787011 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.787528 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.788682 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.788893 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.788967 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.789987 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:11.789990 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:11.789992 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:11.790003 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:11.790021 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:11.790028 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:11.790085 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:11.790215 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 16:03:11.790263 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 16:03:11.791249 14157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.783823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:35641 (local address 127.25.31.131:49527)
0806 16:03:11.784545 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:11.784597 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:11.785906 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:11.786337 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:11.786343 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:11.786421 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:11.786891 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.786902 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.787677 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.787683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.788576 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.788586 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.789019 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.789022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.789161 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.789742 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 16:03:11.789743 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:11.789785 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 16:03:11.789844 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:11.790100 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:11.790107 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:11.790111 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 16:03:11.790198 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 16:03:11.790432 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":140,"threads_started":1}
I20260806 16:03:11.792357 14155 heartbeater.cc:347] Connected to a master server at 127.25.31.190:35641
I20260806 16:03:11.792629 14155 heartbeater.cc:464] Registering TS with master...
I20260806 16:03:11.793125 14155 heartbeater.cc:511] Master 127.25.31.190:35641 requested a full tablet report, sending...
I20260806 16:03:11.794145 13696 ts_manager.cc:194] Registered new tserver with Master: a895ff7c039e4cf3ab331a18d209e495 (127.25.31.131:35089)
I20260806 16:03:11.794749 13696 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.31.131:49527
I20260806 16:03:11.796360 25726 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 16:03:11.801553 13736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.798202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51234 (local address 127.25.31.190:35641)
0806 16:03:11.798358 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:11.798365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:11.798512 (+ 147us) server_negotiation.cc:407] Connection header received
0806 16:03:11.798603 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:11.798605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:11.798640 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:11.798704 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:11.798923 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.799405 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.800181 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.800384 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.800457 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.801130 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:11.801133 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:11.801135 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:11.801145 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:11.801161 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:11.801168 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:11.801223 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:11.801318 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 16:03:11.801364 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 16:03:11.804170 13696 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 16:03:11.807107 13696 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 16:03:11.821413 14171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.815811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49310 (local address 127.25.31.129:37061)
0806 16:03:11.817218 (+ 1407us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:11.817230 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:11.817251 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:03:11.817325 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:11.817329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:11.817497 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:11.817641 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:11.817889 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.818994 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.820236 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.820761 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.820893 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.821076 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:11.821198 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 16:03:11.821281 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1300,"thread_start_us":126,"threads_started":1}
I20260806 16:03:11.821419 14169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.815630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:37061 (local address 127.0.0.1:49310)
0806 16:03:11.816067 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:11.816080 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:11.816201 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:11.817641 (+ 1440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:11.817644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:11.817663 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:11.817752 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.817757 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.819127 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.819134 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.820113 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.820122 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.820887 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.820891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.820996 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.821033 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 16:03:11.821150 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 16:03:11.821266 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":97,"thread_start_us":89,"threads_started":1}
I20260806 16:03:11.822115 14172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.816073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55996 (local address 127.25.31.131:35089)
0806 16:03:11.816766 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:11.816774 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:11.816792 (+ 18us) server_negotiation.cc:407] Connection header received
0806 16:03:11.816849 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:11.816852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:11.816982 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:11.817126 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:11.817353 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.818236 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.820248 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.820764 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.820871 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.821017 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:11.821835 (+ 818us) server_negotiation.cc:299] Negotiation successful
0806 16:03:11.821956 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":147,"threads_started":1}
I20260806 16:03:11.822746 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.815830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.131:35089 (local address 127.0.0.1:55996)
0806 16:03:11.816437 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:11.816471 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:11.816558 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:11.817109 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:11.817113 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:11.817130 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:11.817222 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.817232 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.818378 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.818382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.820105 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.820114 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.821593 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.821599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.821684 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.821705 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 16:03:11.821778 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 16:03:11.822615 (+ 837us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":292,"thread_start_us":99,"threads_started":1}
I20260806 16:03:11.823061 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.815161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:42359 (local address 127.0.0.1:35896)
0806 16:03:11.815517 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:11.815569 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:11.816205 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:11.816892 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:11.816898 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:11.816921 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:11.817061 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.817070 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.818543 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.818546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.820103 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.820114 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.822693 (+ 2579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.822697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.822784 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.822808 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 16:03:11.822875 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 16:03:11.822940 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":159,"thread_start_us":83,"threads_started":1}
I20260806 16:03:11.823343 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.815310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35896 (local address 127.25.31.130:42359)
0806 16:03:11.816468 (+ 1158us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:11.816488 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:11.816513 (+ 25us) server_negotiation.cc:407] Connection header received
0806 16:03:11.816587 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:11.816591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:11.816737 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:11.816909 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:11.817205 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.818422 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.821142 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.822566 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.822672 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.823022 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:11.823067 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 16:03:11.823180 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":123,"threads_started":1}
I20260806 16:03:11.824489 13821 tablet_service.cc:1467] Processing CreateTablet for tablet 4008691dd0e248f38aa4f208b2317f58 (DEFAULT_TABLE table=test-table [id=8632909fe93b4aa8bc7087ece1e40ce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:11.825454 13821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4008691dd0e248f38aa4f208b2317f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:11.825500 13955 tablet_service.cc:1467] Processing CreateTablet for tablet 4008691dd0e248f38aa4f208b2317f58 (DEFAULT_TABLE table=test-table [id=8632909fe93b4aa8bc7087ece1e40ce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:11.826536 13955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4008691dd0e248f38aa4f208b2317f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:11.828279 14088 tablet_service.cc:1467] Processing CreateTablet for tablet 4008691dd0e248f38aa4f208b2317f58 (DEFAULT_TABLE table=test-table [id=8632909fe93b4aa8bc7087ece1e40ce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 16:03:11.829238 14088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4008691dd0e248f38aa4f208b2317f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:11.832471 14173 tablet_bootstrap.cc:492] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb: Bootstrap starting.
I20260806 16:03:11.833719 14174 tablet_bootstrap.cc:492] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f: Bootstrap starting.
I20260806 16:03:11.834791 14173 tablet_bootstrap.cc:654] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:11.835572 14173 log.cc:824] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:11.835808 14174 tablet_bootstrap.cc:654] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:11.836074 14175 tablet_bootstrap.cc:492] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495: Bootstrap starting.
I20260806 16:03:11.836601 14174 log.cc:824] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:11.837996 14175 tablet_bootstrap.cc:654] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:11.838706 14175 log.cc:824] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:11.838994 14173 tablet_bootstrap.cc:492] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb: No bootstrap required, opened a new log
I20260806 16:03:11.839200 14173 ts_tablet_manager.cc:1397] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 16:03:11.839670 14174 tablet_bootstrap.cc:492] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f: No bootstrap required, opened a new log
I20260806 16:03:11.839848 14174 ts_tablet_manager.cc:1397] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 16:03:11.840415 14175 tablet_bootstrap.cc:492] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495: No bootstrap required, opened a new log
I20260806 16:03:11.840602 14175 ts_tablet_manager.cc:1397] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 16:03:11.843165 14173 raft_consensus.cc:348] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe3eb93a5f64765a2af79d3c52b00fb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42359 } } peers { permanent_uuid: "a895ff7c039e4cf3ab331a18d209e495" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35089 } } peers { permanent_uuid: "45bbaf0a2630437e80886cafe93cdd1f" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37061 } }
I20260806 16:03:11.843446 14173 raft_consensus.cc:374] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:11.843497 14173 raft_consensus.cc:729] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffe3eb93a5f64765a2af79d3c52b00fb, State: Initialized, Role: FOLLOWER
I20260806 16:03:11.843822 14175 raft_consensus.cc:348] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe3eb93a5f64765a2af79d3c52b00fb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42359 } } peers { permanent_uuid: "a895ff7c039e4cf3ab331a18d209e495" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35089 } } peers { permanent_uuid: "45bbaf0a2630437e80886cafe93cdd1f" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37061 } }
I20260806 16:03:11.843974 14173 consensus_queue.cc:260] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe3eb93a5f64765a2af79d3c52b00fb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42359 } } peers { permanent_uuid: "a895ff7c039e4cf3ab331a18d209e495" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35089 } } peers { permanent_uuid: "45bbaf0a2630437e80886cafe93cdd1f" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37061 } }
I20260806 16:03:11.844063 14175 raft_consensus.cc:374] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:11.843915 14174 raft_consensus.cc:348] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe3eb93a5f64765a2af79d3c52b00fb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42359 } } peers { permanent_uuid: "a895ff7c039e4cf3ab331a18d209e495" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35089 } } peers { permanent_uuid: "45bbaf0a2630437e80886cafe93cdd1f" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37061 } }
I20260806 16:03:11.844120 14175 raft_consensus.cc:729] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a895ff7c039e4cf3ab331a18d209e495, State: Initialized, Role: FOLLOWER
I20260806 16:03:11.844138 14174 raft_consensus.cc:374] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:11.844187 14174 raft_consensus.cc:729] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45bbaf0a2630437e80886cafe93cdd1f, State: Initialized, Role: FOLLOWER
I20260806 16:03:11.844614 14174 consensus_queue.cc:260] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe3eb93a5f64765a2af79d3c52b00fb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42359 } } peers { permanent_uuid: "a895ff7c039e4cf3ab331a18d209e495" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35089 } } peers { permanent_uuid: "45bbaf0a2630437e80886cafe93cdd1f" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37061 } }
I20260806 16:03:11.844712 14021 heartbeater.cc:503] Master 127.25.31.190:35641 was elected leader, sending a full tablet report...
I20260806 16:03:11.844972 14173 ts_tablet_manager.cc:1428] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 16:03:11.845353 13889 heartbeater.cc:503] Master 127.25.31.190:35641 was elected leader, sending a full tablet report...
I20260806 16:03:11.845570 14174 ts_tablet_manager.cc:1428] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 16:03:11.845851 14175 consensus_queue.cc:260] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe3eb93a5f64765a2af79d3c52b00fb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42359 } } peers { permanent_uuid: "a895ff7c039e4cf3ab331a18d209e495" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35089 } } peers { permanent_uuid: "45bbaf0a2630437e80886cafe93cdd1f" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37061 } }
I20260806 16:03:11.846511 14155 heartbeater.cc:503] Master 127.25.31.190:35641 was elected leader, sending a full tablet report...
I20260806 16:03:11.846791 14175 ts_tablet_manager.cc:1428] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 16:03:11.873121 14181 raft_consensus.cc:482] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 16:03:11.873287 14181 raft_consensus.cc:504] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe3eb93a5f64765a2af79d3c52b00fb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42359 } } peers { permanent_uuid: "a895ff7c039e4cf3ab331a18d209e495" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35089 } } peers { permanent_uuid: "45bbaf0a2630437e80886cafe93cdd1f" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37061 } }
I20260806 16:03:11.874459 14181 leader_election.cc:288] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffe3eb93a5f64765a2af79d3c52b00fb (127.25.31.130:42359), 45bbaf0a2630437e80886cafe93cdd1f (127.25.31.129:37061)
I20260806 16:03:11.879148 14157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.874659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:37061 (local address 127.25.31.131:52591)
0806 16:03:11.874839 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:11.874856 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:11.874948 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:11.875245 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:11.875251 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:11.875275 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:11.875417 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.875427 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.876380 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.876384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.877689 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.877699 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.878650 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.878652 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.878756 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.878820 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 16:03:11.878908 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 16:03:11.878992 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":21}
I20260806 16:03:11.879158 14171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.874856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:52591 (local address 127.25.31.129:37061)
0806 16:03:11.875003 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:11.875008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:11.875024 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:03:11.875076 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:11.875079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:11.875137 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:11.875217 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:11.875541 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.876252 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.877809 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.878534 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.878656 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.878719 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:11.878935 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 16:03:11.879033 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 16:03:11.879223 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.874773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.31.131:53649 (local address 127.25.31.130:42359)
0806 16:03:11.874906 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:11.874910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:11.875291 (+ 381us) server_negotiation.cc:407] Connection header received
0806 16:03:11.875523 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:11.875526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:11.875581 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:11.875661 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 16:03:11.875971 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.876644 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.878069 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.878662 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.878769 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.878812 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:11.879026 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 16:03:11.879104 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 16:03:11.879174 14182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.874657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:42359 (local address 127.25.31.131:53649)
0806 16:03:11.875209 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:11.875222 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:11.875363 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:11.875741 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:11.875744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:11.875761 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 16:03:11.875862 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.875867 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.876739 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.876743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.877860 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:11.877873 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.878767 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.878770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:11.878912 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.878933 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 16:03:11.878999 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 16:03:11.879047 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":209,"thread_start_us":122,"threads_started":1}
I20260806 16:03:11.879873 13975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4008691dd0e248f38aa4f208b2317f58" candidate_uuid: "a895ff7c039e4cf3ab331a18d209e495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffe3eb93a5f64765a2af79d3c52b00fb" is_pre_election: true
I20260806 16:03:11.879874 13841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4008691dd0e248f38aa4f208b2317f58" candidate_uuid: "a895ff7c039e4cf3ab331a18d209e495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45bbaf0a2630437e80886cafe93cdd1f" is_pre_election: true
I20260806 16:03:11.880174 13975 raft_consensus.cc:2454] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a895ff7c039e4cf3ab331a18d209e495 in term 0.
I20260806 16:03:11.880174 13841 raft_consensus.cc:2454] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a895ff7c039e4cf3ab331a18d209e495 in term 0.
I20260806 16:03:11.880647 14043 leader_election.cc:302] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45bbaf0a2630437e80886cafe93cdd1f, a895ff7c039e4cf3ab331a18d209e495; no voters:
I20260806 16:03:11.880916 14181 raft_consensus.cc:2781] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 16:03:11.880992 14181 raft_consensus.cc:482] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 16:03:11.881023 14181 raft_consensus.cc:3037] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:11.882082 14181 raft_consensus.cc:504] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe3eb93a5f64765a2af79d3c52b00fb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42359 } } peers { permanent_uuid: "a895ff7c039e4cf3ab331a18d209e495" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35089 } } peers { permanent_uuid: "45bbaf0a2630437e80886cafe93cdd1f" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37061 } }
I20260806 16:03:11.882475 14181 leader_election.cc:288] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [CANDIDATE]: Term 1 election: Requested vote from peers ffe3eb93a5f64765a2af79d3c52b00fb (127.25.31.130:42359), 45bbaf0a2630437e80886cafe93cdd1f (127.25.31.129:37061)
I20260806 16:03:11.882846 13841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4008691dd0e248f38aa4f208b2317f58" candidate_uuid: "a895ff7c039e4cf3ab331a18d209e495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45bbaf0a2630437e80886cafe93cdd1f"
I20260806 16:03:11.882833 13975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4008691dd0e248f38aa4f208b2317f58" candidate_uuid: "a895ff7c039e4cf3ab331a18d209e495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffe3eb93a5f64765a2af79d3c52b00fb"
I20260806 16:03:11.882965 13841 raft_consensus.cc:3037] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:11.882965 13975 raft_consensus.cc:3037] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:11.884217 13975 raft_consensus.cc:2454] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a895ff7c039e4cf3ab331a18d209e495 in term 1.
I20260806 16:03:11.884217 13841 raft_consensus.cc:2454] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a895ff7c039e4cf3ab331a18d209e495 in term 1.
I20260806 16:03:11.884543 14041 leader_election.cc:302] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a895ff7c039e4cf3ab331a18d209e495, ffe3eb93a5f64765a2af79d3c52b00fb; no voters:
I20260806 16:03:11.884704 14181 raft_consensus.cc:2781] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:11.884917 14181 raft_consensus.cc:686] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [term 1 LEADER]: Becoming Leader. State: Replica: a895ff7c039e4cf3ab331a18d209e495, State: Running, Role: LEADER
I20260806 16:03:11.885227 14181 consensus_queue.cc:237] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe3eb93a5f64765a2af79d3c52b00fb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42359 } } peers { permanent_uuid: "a895ff7c039e4cf3ab331a18d209e495" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35089 } } peers { permanent_uuid: "45bbaf0a2630437e80886cafe93cdd1f" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37061 } }
I20260806 16:03:11.888201 13694 catalog_manager.cc:5495] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 reported cstate change: term changed from 0 to 1, leader changed from <none> to a895ff7c039e4cf3ab331a18d209e495 (127.25.31.131). New cstate: current_term: 1 leader_uuid: "a895ff7c039e4cf3ab331a18d209e495" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffe3eb93a5f64765a2af79d3c52b00fb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a895ff7c039e4cf3ab331a18d209e495" member_type: VOTER last_known_addr { host: "127.25.31.131" port: 35089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45bbaf0a2630437e80886cafe93cdd1f" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37061 } health_report { overall_health: UNKNOWN } } }
I20260806 16:03:11.898108 14172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.894804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56008 (local address 127.25.31.131:35089)
0806 16:03:11.894924 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:11.894928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:11.894940 (+ 12us) server_negotiation.cc:407] Connection header received
0806 16:03:11.895029 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:11.895031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:11.895077 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:11.895157 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:11.895352 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.895823 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.896672 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:11.896874 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:11.896956 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:11.897302 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:11.897728 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:11.897869 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:11.897903 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 16:03:11.897954 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 16:03:11.903513 13841 raft_consensus.cc:1261] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f [term 1 FOLLOWER]: Refusing update from remote peer a895ff7c039e4cf3ab331a18d209e495: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:11.903515 13975 raft_consensus.cc:1261] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb [term 1 FOLLOWER]: Refusing update from remote peer a895ff7c039e4cf3ab331a18d209e495: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:11.904150 14181 consensus_queue.cc:1035] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffe3eb93a5f64765a2af79d3c52b00fb" member_type: VOTER last_known_addr { host: "127.25.31.130" port: 42359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:03:11.904343 14183 consensus_queue.cc:1035] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45bbaf0a2630437e80886cafe93cdd1f" member_type: VOTER last_known_addr { host: "127.25.31.129" port: 37061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 16:03:11.908684 13890 tablet.cc:2369] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:03:11.912395 14187 mvcc.cc:205] Tried to move back new op lower bound from 7315587858029805568 to 7315587857964494848. Current Snapshot: MvccSnapshot[applied={T|T < 7315587858029805568}]
I20260806 16:03:11.912942 14188 mvcc.cc:205] Tried to move back new op lower bound from 7315587858029805568 to 7315587857964494848. Current Snapshot: MvccSnapshot[applied={T|T < 7315587858029805568}]
I20260806 16:03:11.920600 13696 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:03:11.920779 13696 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 16:03:11.923230 13696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c9940c26bd5479ca2383c4979d75e7b: Sending DeleteTablet for 3 replicas of tablet 4008691dd0e248f38aa4f208b2317f58
I20260806 16:03:11.924021 13821 tablet_service.cc:1514] Processing DeleteTablet for tablet 4008691dd0e248f38aa4f208b2317f58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:11 UTC) from {username='slave'} at 127.0.0.1:49310
I20260806 16:03:11.924160 13955 tablet_service.cc:1514] Processing DeleteTablet for tablet 4008691dd0e248f38aa4f208b2317f58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:11 UTC) from {username='slave'} at 127.0.0.1:35896
I20260806 16:03:11.924476 14195 tablet_replica.cc:330] T 4008691dd0e248f38aa4f208b2317f58 P 45bbaf0a2630437e80886cafe93cdd1f: stopping tablet replica
I20260806 16:03:11.924712 14088 tablet_service.cc:1514] Processing DeleteTablet for tablet 4008691dd0e248f38aa4f208b2317f58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 16:03:11 UTC) from {username='slave'} at 127.0.0.1:55996
I20260806 16:03:11.924924 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 13757
I20260806 16:03:11.926422 14197 tablet_replica.cc:330] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495: stopping tablet replica
I20260806 16:03:11.926720 14197 raft_consensus.cc:2229] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [term 1 LEADER]: Raft consensus shutting down.
I20260806 16:03:11.927142 14197 raft_consensus.cc:2258] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:03:11.928778 14197 ts_tablet_manager.cc:1905] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:03:11.930541 14196 tablet_replica.cc:330] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb: stopping tablet replica
I20260806 16:03:11.930912 14196 raft_consensus.cc:2229] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 16:03:11.931278 14196 raft_consensus.cc:2258] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 16:03:11.932407 14197 ts_tablet_manager.cc:1918] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 16:03:11.932516 14197 log.cc:1196] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495: Deleting WAL directory at /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786031939584994-25726-0/minicluster-data/ts-2/wal/wals/4008691dd0e248f38aa4f208b2317f58
I20260806 16:03:11.932842 14197 ts_tablet_manager.cc:1939] T 4008691dd0e248f38aa4f208b2317f58 P a895ff7c039e4cf3ab331a18d209e495: Deleting consensus metadata
I20260806 16:03:11.933115 14196 ts_tablet_manager.cc:1905] T 4008691dd0e248f38aa4f208b2317f58 P ffe3eb93a5f64765a2af79d3c52b00fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 16:03:11.934161 13681 catalog_manager.cc:4841] TS a895ff7c039e4cf3ab331a18d209e495 (127.25.31.131:35089): tablet 4008691dd0e248f38aa4f208b2317f58 (table test-table [id=8632909fe93b4aa8bc7087ece1e40ce9]) successfully deleted
W20260806 16:03:11.934711 13683 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.31.129:37061 (error 108)
I20260806 16:03:11.934962 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 13892
I20260806 16:03:11.937103 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.936807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.129:37061 (local address 127.0.0.1:49326)
0806 16:03:11.936953 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:11.937031 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.129:37061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 16:03:11.937325 13683 catalog_manager.cc:4584] TS 45bbaf0a2630437e80886cafe93cdd1f (127.25.31.129:37061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4008691dd0e248f38aa4f208b2317f58: Network error: Client connection negotiation failed: client connection to 127.25.31.129:37061: connect: Connection refused (error 111)
I20260806 16:03:11.944307 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 14024
I20260806 16:03:11.946724 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:11.946272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.130:42359 (local address 127.0.0.1:35904)
0806 16:03:11.946519 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:11.946594 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.130:42359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260806 16:03:11.952831 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 13665
2026-08-06T16:03:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (997 ms)
[----------] 6 tests from EncryptionPolicyTest (10141 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:12 dist-test-slave-tm1g krb5kdc[14201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:12 dist-test-slave-tm1g krb5kdc[14201](info): set up 2 sockets
Aug 06 16:03:12 dist-test-slave-tm1g krb5kdc[14201](info): commencing operation
krb5kdc: starting...
W20260806 16:03:12.998456 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:13 dist-test-slave-tm1g krb5kdc[14201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:03:13.117848 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:13.225015 14214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:13.225319 14214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:13.225404 14214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:13.229372 14214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:13.229461 14214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:13.229501 14214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:13.229557 14214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:13.229597 14214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:13.234637 14214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35029
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:13.235599 14214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:13.236565 14214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:13.244750 14220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:13.244765 14219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:13.244767 14222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:13.244829 14214 server_base.cc:1034] running on GCE node
I20260806 16:03:13.245569 14214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:13.246658 14214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:13.247849 14214 hybrid_clock.cc:648] HybridClock initialized: now 1786032193247831 us; error 31 us; skew 500 ppm
Aug 06 16:03:13 dist-test-slave-tm1g krb5kdc[14201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032193, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:13.250573 14214 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:03:13.251448 14214 webserver.cc:460] Webserver started at http://127.0.0.1:39629/ using document root <none> and password file <none>
I20260806 16:03:13.251982 14214 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:13.252095 14214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:13.252333 14214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:13.254285 14214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "80d6337792674d10aefb0bc4a1604a58"
format_stamp: "Formatted at 2026-08-06 16:03:13 on dist-test-slave-tm1g"
server_key: "2792856eda95b2d1d734a0faea98ea1b"
server_key_iv: "845881ab7feb102e41328fcc32dc2f23"
server_key_version: "encryptionkey@0"
I20260806 16:03:13.254904 14214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "80d6337792674d10aefb0bc4a1604a58"
format_stamp: "Formatted at 2026-08-06 16:03:13 on dist-test-slave-tm1g"
server_key: "2792856eda95b2d1d734a0faea98ea1b"
server_key_iv: "845881ab7feb102e41328fcc32dc2f23"
server_key_version: "encryptionkey@0"
I20260806 16:03:13.258400 14214 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 16:03:13.260675 14230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:13.261821 14214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:03:13.261960 14214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "80d6337792674d10aefb0bc4a1604a58"
format_stamp: "Formatted at 2026-08-06 16:03:13 on dist-test-slave-tm1g"
server_key: "2792856eda95b2d1d734a0faea98ea1b"
server_key_iv: "845881ab7feb102e41328fcc32dc2f23"
server_key_version: "encryptionkey@0"
I20260806 16:03:13.262069 14214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:13.278995 14214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:13.281612 14214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:13.281803 14214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:13.288728 14214 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35029
I20260806 16:03:13.288756 14282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35029 every 8 connection(s)
I20260806 16:03:13.289633 14214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:13.292328 14283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:13.293529 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 14214
I20260806 16:03:13.293676 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:13.294008 25726 external_mini_cluster.cc:1428] Setting key 0db8af44f0bf98fbfd1e8ad0c0b2c031
I20260806 16:03:13.298990 14283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58: Bootstrap starting.
I20260806 16:03:13.301501 14283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:13.302556 14283 log.cc:824] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:03:13 dist-test-slave-tm1g krb5kdc[14201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 16:03:13.304689 14283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58: No bootstrap required, opened a new log
I20260806 16:03:13.309453 14286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:13.295455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34837 (local address 127.0.0.1:35029)
0806 16:03:13.295878 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:13.295897 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:13.295939 (+ 42us) server_negotiation.cc:407] Connection header received
0806 16:03:13.296749 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:13.296772 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:13.297205 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:13.297559 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:13.298223 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:13.299660 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:13.300517 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:13.301188 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:13.301360 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:13.304080 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:13.304086 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:13.304100 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:13.304131 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:13.306039 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:13.306605 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:13.306611 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:13.306617 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:13.306754 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:13.307096 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:13.307102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:13.307105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:13.307641 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:13.307866 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:13.308082 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 16:03:13.308493 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":109,"threads_started":1}
I20260806 16:03:13.309458 14283 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d6337792674d10aefb0bc4a1604a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35029 } }
I20260806 16:03:13.309777 14283 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:13.309842 14283 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80d6337792674d10aefb0bc4a1604a58, State: Initialized, Role: FOLLOWER
I20260806 16:03:13.310241 14283 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d6337792674d10aefb0bc4a1604a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35029 } }
I20260806 16:03:13.310431 14283 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:13.310498 14283 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:13.310604 14283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:13.311810 14283 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d6337792674d10aefb0bc4a1604a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35029 } }
I20260806 16:03:13.312186 14283 leader_election.cc:302] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80d6337792674d10aefb0bc4a1604a58; no voters:
I20260806 16:03:13.312529 14283 leader_election.cc:288] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:13.312695 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:13.312731 14288 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:13.312984 14288 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [term 1 LEADER]: Becoming Leader. State: Replica: 80d6337792674d10aefb0bc4a1604a58, State: Running, Role: LEADER
I20260806 16:03:13.313323 14288 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d6337792674d10aefb0bc4a1604a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35029 } }
I20260806 16:03:13.313823 14283 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:13.314849 14289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80d6337792674d10aefb0bc4a1604a58. Latest consensus state: current_term: 1 leader_uuid: "80d6337792674d10aefb0bc4a1604a58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d6337792674d10aefb0bc4a1604a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35029 } } }
I20260806 16:03:13.315063 14289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:13.315218 14290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80d6337792674d10aefb0bc4a1604a58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80d6337792674d10aefb0bc4a1604a58" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35029 } } }
I20260806 16:03:13.315346 14290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:13.315516 14301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:13.318545 14301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:13.324923 14286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:13.314949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58587 (local address 127.0.0.1:35029)
0806 16:03:13.315076 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:13.315081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:13.315762 (+ 681us) server_negotiation.cc:407] Connection header received
0806 16:03:13.315949 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:13.315954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:13.316003 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:13.316072 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:13.318096 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:13.318730 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:13.319900 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:13.320145 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:13.320250 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:13.321628 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:13.321632 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:13.321635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:13.321659 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:13.323142 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:13.323620 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:13.323624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:13.323625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:13.323707 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:13.324043 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:13.324047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:13.324048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:13.324220 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:13.324368 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:13.324644 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 16:03:13.324735 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 16:03:13.329527 14301 catalog_manager.cc:1347] Generated new cluster ID: ae420d18bacc45bdb78e05659dc497d2
I20260806 16:03:13.329600 14301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:13.347456 14301 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:13.348351 14301 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:13.351890 14301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80d6337792674d10aefb0bc4a1604a58: Generated new TSK 0
I20260806 16:03:13.352505 14301 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:13.357254 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 14214
W20260806 16:03:13.363971 14199 connection.cc:535] client connection to 127.0.0.1:35029 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35029): Connection reset by peer (error 104)
2026-08-06T16:03:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1402 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:13 dist-test-slave-tm1g krb5kdc[14314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:13 dist-test-slave-tm1g krb5kdc[14314](info): set up 2 sockets
Aug 06 16:03:13 dist-test-slave-tm1g krb5kdc[14314](info): commencing operation
krb5kdc: starting...
W20260806 16:03:14.407120 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:14 dist-test-slave-tm1g krb5kdc[14314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:03:14.524762 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39173
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:14.631335 14327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:14.631577 14327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:14.631618 14327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:14.635537 14327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:14.635589 14327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:14.635609 14327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:14.635672 14327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:14.635705 14327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:14.640808 14327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39173
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:14.641790 14327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:14.642786 14327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:14.650782 14333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:14.650782 14332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:14.650803 14327 server_base.cc:1034] running on GCE node
W20260806 16:03:14.650815 14335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:14.651675 14327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:14.652695 14327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:14.653859 14327 hybrid_clock.cc:648] HybridClock initialized: now 1786032194653841 us; error 29 us; skew 500 ppm
Aug 06 16:03:14 dist-test-slave-tm1g krb5kdc[14314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032194, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:14.656502 14327 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:03:14.657330 14327 webserver.cc:460] Webserver started at http://127.0.0.1:33597/ using document root <none> and password file <none>
I20260806 16:03:14.657859 14327 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:14.657919 14327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:14.658154 14327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:14.660202 14327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "d7687ef38b5c44e18e95200f7c1ec66b"
format_stamp: "Formatted at 2026-08-06 16:03:14 on dist-test-slave-tm1g"
server_key: "e18f909dd12f10cc69f0bf4472c7b09c"
server_key_iv: "0fd9a3ae45ed79ccfac40bb4f33b5549"
server_key_version: "encryptionkey@0"
I20260806 16:03:14.660756 14327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "d7687ef38b5c44e18e95200f7c1ec66b"
format_stamp: "Formatted at 2026-08-06 16:03:14 on dist-test-slave-tm1g"
server_key: "e18f909dd12f10cc69f0bf4472c7b09c"
server_key_iv: "0fd9a3ae45ed79ccfac40bb4f33b5549"
server_key_version: "encryptionkey@0"
I20260806 16:03:14.664296 14327 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 16:03:14.666561 14342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:14.667719 14327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 16:03:14.667861 14327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "d7687ef38b5c44e18e95200f7c1ec66b"
format_stamp: "Formatted at 2026-08-06 16:03:14 on dist-test-slave-tm1g"
server_key: "e18f909dd12f10cc69f0bf4472c7b09c"
server_key_iv: "0fd9a3ae45ed79ccfac40bb4f33b5549"
server_key_version: "encryptionkey@0"
I20260806 16:03:14.667965 14327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:14.688850 14327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:14.691599 14327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:14.691797 14327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:14.698558 14327 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39173
I20260806 16:03:14.698575 14395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39173 every 8 connection(s)
I20260806 16:03:14.699455 14327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:14.700443 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 14327
I20260806 16:03:14.700603 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:14.700899 25726 external_mini_cluster.cc:1428] Setting key cba5bab7fb053ae643da956e58ed9ab6
I20260806 16:03:14.702247 14396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:14.708307 14396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b: Bootstrap starting.
I20260806 16:03:14.709715 14399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:14.702247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41455 (local address 127.0.0.1:39173)
0806 16:03:14.702688 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:14.702703 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:14.702736 (+ 33us) server_negotiation.cc:407] Connection header received
0806 16:03:14.703364 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:14.703379 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:14.703690 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:14.703953 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:14.704343 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:14.705350 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:14.706582 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:14.707031 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:14.707155 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:14.708053 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:14.708058 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:14.708069 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:14.708466 (+ 397us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:14.708536 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:14.708563 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:14.708663 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:14.708849 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 16:03:14.708996 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":144,"threads_started":1}
I20260806 16:03:14.710631 14396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:14.711329 14396 log.cc:824] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:14.712507 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:14.713445 14396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b: No bootstrap required, opened a new log
I20260806 16:03:14.717939 14396 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7687ef38b5c44e18e95200f7c1ec66b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39173 } }
I20260806 16:03:14.718182 14396 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:14.718256 14396 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7687ef38b5c44e18e95200f7c1ec66b, State: Initialized, Role: FOLLOWER
I20260806 16:03:14.718737 14396 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7687ef38b5c44e18e95200f7c1ec66b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39173 } }
I20260806 16:03:14.718876 14399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:14.714822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39877 (local address 127.0.0.1:39173)
0806 16:03:14.714946 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:14.714950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:14.715105 (+ 155us) server_negotiation.cc:407] Connection header received
0806 16:03:14.715196 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:14.715199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:14.715243 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:14.715320 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:14.715557 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:14.716391 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:14.717198 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:14.717550 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:14.717657 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:14.718471 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:14.718476 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:14.718480 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:14.718492 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:14.718517 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:14.718530 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:14.718625 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:14.718720 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 16:03:14.718756 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 16:03:14.718897 14396 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:14.719085 14396 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:14.719218 14396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:14.720193 14396 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7687ef38b5c44e18e95200f7c1ec66b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39173 } }
I20260806 16:03:14.720522 14396 leader_election.cc:302] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7687ef38b5c44e18e95200f7c1ec66b; no voters:
I20260806 16:03:14.720849 14396 leader_election.cc:288] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:14.720963 14406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:14.721189 14406 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [term 1 LEADER]: Becoming Leader. State: Replica: d7687ef38b5c44e18e95200f7c1ec66b, State: Running, Role: LEADER
I20260806 16:03:14.721544 14406 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7687ef38b5c44e18e95200f7c1ec66b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39173 } }
I20260806 16:03:14.722047 14396 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:14.723572 14407 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7687ef38b5c44e18e95200f7c1ec66b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7687ef38b5c44e18e95200f7c1ec66b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39173 } } }
I20260806 16:03:14.723721 14407 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:14.724030 14408 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7687ef38b5c44e18e95200f7c1ec66b. Latest consensus state: current_term: 1 leader_uuid: "d7687ef38b5c44e18e95200f7c1ec66b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7687ef38b5c44e18e95200f7c1ec66b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39173 } } }
I20260806 16:03:14.724164 14408 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:14.724427 14418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:14.727265 14418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:14.732308 14418 catalog_manager.cc:1347] Generated new cluster ID: eddab096b394452ca31f7a3922afc6e9
I20260806 16:03:14.732378 14418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:14.750464 14418 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:14.751359 14418 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:14.757268 14418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7687ef38b5c44e18e95200f7c1ec66b: Generated new TSK 0
I20260806 16:03:14.757903 14418 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:14.766057 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 14327
W20260806 16:03:14.772177 14312 connection.cc:535] client connection to 127.0.0.1:39173 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39173): Connection reset by peer (error 104)
2026-08-06T16:03:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1408 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:14 dist-test-slave-tm1g krb5kdc[14427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:14 dist-test-slave-tm1g krb5kdc[14427](info): set up 2 sockets
Aug 06 16:03:14 dist-test-slave-tm1g krb5kdc[14427](info): commencing operation
krb5kdc: starting...
W20260806 16:03:15.810982 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:15 dist-test-slave-tm1g krb5kdc[14427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:03:15.928970 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39489
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:16.037552 14440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:16.037801 14440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:16.037842 14440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:16.041796 14440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:16.041851 14440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:16.041870 14440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:16.041886 14440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:16.041949 14440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:16.047147 14440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39489
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:16.048139 14440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:16.049120 14440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:16.057528 14446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:16.057567 14445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:16.057560 14448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:16.057624 14440 server_base.cc:1034] running on GCE node
I20260806 16:03:16.058295 14440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:16.059336 14440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:16.060509 14440 hybrid_clock.cc:648] HybridClock initialized: now 1786032196060490 us; error 30 us; skew 500 ppm
Aug 06 16:03:16 dist-test-slave-tm1g krb5kdc[14427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:16.067648 14440 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:03:16.068457 14440 webserver.cc:460] Webserver started at http://127.0.0.1:41225/ using document root <none> and password file <none>
I20260806 16:03:16.068955 14440 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:16.069021 14440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:16.069185 14440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:16.071233 14440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "2de293cda83e4b5daf4015c1fc941c19"
format_stamp: "Formatted at 2026-08-06 16:03:16 on dist-test-slave-tm1g"
server_key: "f5ad57c77801af575d380e059016d834"
server_key_iv: "ca41d2485bfdafc1e34bf67aa296aa98"
server_key_version: "encryptionkey@0"
I20260806 16:03:16.071732 14440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "2de293cda83e4b5daf4015c1fc941c19"
format_stamp: "Formatted at 2026-08-06 16:03:16 on dist-test-slave-tm1g"
server_key: "f5ad57c77801af575d380e059016d834"
server_key_iv: "ca41d2485bfdafc1e34bf67aa296aa98"
server_key_version: "encryptionkey@0"
I20260806 16:03:16.075285 14440 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:03:16.077567 14455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:16.078961 14440 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:03:16.079077 14440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "2de293cda83e4b5daf4015c1fc941c19"
format_stamp: "Formatted at 2026-08-06 16:03:16 on dist-test-slave-tm1g"
server_key: "f5ad57c77801af575d380e059016d834"
server_key_iv: "ca41d2485bfdafc1e34bf67aa296aa98"
server_key_version: "encryptionkey@0"
I20260806 16:03:16.079161 14440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:16.083560 14440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:16.086725 14440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:16.086865 14440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:16.093771 14440 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39489
I20260806 16:03:16.093794 14507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39489 every 8 connection(s)
I20260806 16:03:16.094734 14440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:16.097328 14508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:16.102635 14508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19: Bootstrap starting.
I20260806 16:03:16.104595 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 14440
I20260806 16:03:16.104768 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:16.105098 25726 external_mini_cluster.cc:1428] Setting key df877ded522b857d7712242fba3cf21e
I20260806 16:03:16.105315 14508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:16.106237 14508 log.cc:824] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:16.108242 14508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19: No bootstrap required, opened a new log
I20260806 16:03:16.109966 14511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:16.106198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54353 (local address 127.0.0.1:39489)
0806 16:03:16.106700 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:16.106716 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:16.106750 (+ 34us) server_negotiation.cc:407] Connection header received
0806 16:03:16.107393 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:16.107409 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:16.107663 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:16.107931 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:16.108798 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:16.108802 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:16.108813 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:16.108823 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:16.108876 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:16.108894 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:16.108977 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:16.109125 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 16:03:16.109290 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":176,"threads_started":1}
I20260806 16:03:16.112181 14508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2de293cda83e4b5daf4015c1fc941c19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39489 } }
I20260806 16:03:16.112417 14508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:16.112478 14508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2de293cda83e4b5daf4015c1fc941c19, State: Initialized, Role: FOLLOWER
I20260806 16:03:16.112594 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:16.112941 14508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2de293cda83e4b5daf4015c1fc941c19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39489 } }
I20260806 16:03:16.113101 14508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:16.113168 14508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:16.113312 14508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:16.114184 14508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2de293cda83e4b5daf4015c1fc941c19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39489 } }
I20260806 16:03:16.114547 14508 leader_election.cc:302] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2de293cda83e4b5daf4015c1fc941c19; no voters:
I20260806 16:03:16.114873 14508 leader_election.cc:288] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:16.114970 14517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:16.115231 14517 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [term 1 LEADER]: Becoming Leader. State: Replica: 2de293cda83e4b5daf4015c1fc941c19, State: Running, Role: LEADER
I20260806 16:03:16.115518 14517 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2de293cda83e4b5daf4015c1fc941c19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39489 } }
I20260806 16:03:16.116274 14508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:16.117534 14519 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2de293cda83e4b5daf4015c1fc941c19. Latest consensus state: current_term: 1 leader_uuid: "2de293cda83e4b5daf4015c1fc941c19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2de293cda83e4b5daf4015c1fc941c19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39489 } } }
I20260806 16:03:16.117542 14520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2de293cda83e4b5daf4015c1fc941c19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2de293cda83e4b5daf4015c1fc941c19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39489 } } }
I20260806 16:03:16.117738 14520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:16.117957 14519 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:16.118508 14527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:16.118724 14511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:16.114827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42435 (local address 127.0.0.1:39489)
0806 16:03:16.114968 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:16.114972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:16.116995 (+ 2023us) server_negotiation.cc:407] Connection header received
0806 16:03:16.117197 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:16.117202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:16.117243 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:16.117303 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:16.118276 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:16.118280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:16.118284 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:16.118319 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:16.118356 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:16.118371 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:16.118458 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:16.118576 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 16:03:16.118617 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 16:03:16.122605 14527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:16.127537 14527 catalog_manager.cc:1347] Generated new cluster ID: 8bce8a6da5aa44c8bab7cc9babc10257
I20260806 16:03:16.127597 14527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:16.136818 14527 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:16.137128 14527 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:16.142623 14527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2de293cda83e4b5daf4015c1fc941c19: Generated new TSK 0
I20260806 16:03:16.143316 14527 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:16.149573 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 14440
2026-08-06T16:03:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1383 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:16 dist-test-slave-tm1g krb5kdc[14539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:16 dist-test-slave-tm1g krb5kdc[14539](info): set up 2 sockets
Aug 06 16:03:16 dist-test-slave-tm1g krb5kdc[14539](info): commencing operation
krb5kdc: starting...
W20260806 16:03:17.210740 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:17 dist-test-slave-tm1g krb5kdc[14539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:03:17.329862 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37173
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:17.439245 14553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:17.439536 14553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:17.439612 14553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:17.443535 14553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:17.443630 14553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:17.443670 14553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:17.443707 14553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:17.443745 14553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:17.448916 14553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:17.449947 14553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:17.450954 14553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:17.459228 14559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:17.459273 14558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:17.459239 14561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:17.459455 14553 server_base.cc:1034] running on GCE node
I20260806 16:03:17.460091 14553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:17.461108 14553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:17.462353 14553 hybrid_clock.cc:648] HybridClock initialized: now 1786032197462275 us; error 43 us; skew 500 ppm
Aug 06 16:03:17 dist-test-slave-tm1g krb5kdc[14539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032197, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:17.465047 14553 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:03:17.465919 14553 webserver.cc:460] Webserver started at http://127.0.0.1:41731/ using document root <none> and password file <none>
I20260806 16:03:17.466549 14553 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:17.466641 14553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:17.466872 14553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:17.468849 14553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "62cfdc30084c480f97d23c785477597b"
format_stamp: "Formatted at 2026-08-06 16:03:17 on dist-test-slave-tm1g"
server_key: "305c3aebd9a07aa36f519e115dfa4f2f"
server_key_iv: "0f363e25d1facc4b175982cc2393799f"
server_key_version: "encryptionkey@0"
I20260806 16:03:17.469445 14553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "62cfdc30084c480f97d23c785477597b"
format_stamp: "Formatted at 2026-08-06 16:03:17 on dist-test-slave-tm1g"
server_key: "305c3aebd9a07aa36f519e115dfa4f2f"
server_key_iv: "0f363e25d1facc4b175982cc2393799f"
server_key_version: "encryptionkey@0"
I20260806 16:03:17.473037 14553 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 16:03:17.475412 14568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:17.476596 14553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:03:17.476735 14553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "62cfdc30084c480f97d23c785477597b"
format_stamp: "Formatted at 2026-08-06 16:03:17 on dist-test-slave-tm1g"
server_key: "305c3aebd9a07aa36f519e115dfa4f2f"
server_key_iv: "0f363e25d1facc4b175982cc2393799f"
server_key_version: "encryptionkey@0"
I20260806 16:03:17.476845 14553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:17.498126 14553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:17.501233 14553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:17.501475 14553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:17.508620 14553 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37173
I20260806 16:03:17.508647 14621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37173 every 8 connection(s)
I20260806 16:03:17.509559 14553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:17.512193 14622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:17.515878 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 14553
I20260806 16:03:17.516026 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:17.516299 25726 external_mini_cluster.cc:1428] Setting key 1a7610c1f38a5089457bb43b77d06505
I20260806 16:03:17.519608 14622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b: Bootstrap starting.
I20260806 16:03:17.522267 14622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:17.523133 14622 log.cc:824] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:17.524943 14622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b: No bootstrap required, opened a new log
Aug 06 16:03:17 dist-test-slave-tm1g krb5kdc[14539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 16:03:17.527976 14622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62cfdc30084c480f97d23c785477597b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37173 } }
I20260806 16:03:17.528210 14622 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:17.528288 14622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62cfdc30084c480f97d23c785477597b, State: Initialized, Role: FOLLOWER
I20260806 16:03:17.528734 14622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62cfdc30084c480f97d23c785477597b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37173 } }
I20260806 16:03:17.528880 14622 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:17.528967 14622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:17.529086 14622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:17.529958 14622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62cfdc30084c480f97d23c785477597b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37173 } }
I20260806 16:03:17.530287 14622 leader_election.cc:302] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62cfdc30084c480f97d23c785477597b; no voters:
I20260806 16:03:17.530689 14622 leader_election.cc:288] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:17.530809 14628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:17.531023 14628 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [term 1 LEADER]: Becoming Leader. State: Replica: 62cfdc30084c480f97d23c785477597b, State: Running, Role: LEADER
I20260806 16:03:17.531385 14628 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62cfdc30084c480f97d23c785477597b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37173 } }
I20260806 16:03:17.531991 14622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:17.532516 14626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:17.517655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56909 (local address 127.0.0.1:37173)
0806 16:03:17.518092 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:17.518112 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:17.518158 (+ 46us) server_negotiation.cc:407] Connection header received
0806 16:03:17.519003 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:17.519026 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:17.519449 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:17.519811 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:17.520471 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:17.521957 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:17.522903 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:17.523604 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:17.523788 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:17.526383 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:17.526389 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:17.526402 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:17.526428 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:17.528904 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:17.529459 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:17.529465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:17.529471 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:17.529596 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:17.529885 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:17.529889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:17.529891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:17.530419 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:17.530640 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:17.531276 (+ 636us) server_negotiation.cc:299] Negotiation successful
0806 16:03:17.531571 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":110,"threads_started":1}
I20260806 16:03:17.533107 14630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62cfdc30084c480f97d23c785477597b. Latest consensus state: current_term: 1 leader_uuid: "62cfdc30084c480f97d23c785477597b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62cfdc30084c480f97d23c785477597b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37173 } } }
I20260806 16:03:17.533250 14630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:17.533588 14629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62cfdc30084c480f97d23c785477597b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62cfdc30084c480f97d23c785477597b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37173 } } }
I20260806 16:03:17.533668 14629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:17.534461 14637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:17.535836 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:17.537484 14637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:17.544055 14637 catalog_manager.cc:1347] Generated new cluster ID: 1f625d3cc1ff4bc0898d4a357656eb8e
I20260806 16:03:17.544116 14637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:17.546432 14626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:17.537859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50181 (local address 127.0.0.1:37173)
0806 16:03:17.537970 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:17.537975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:17.538215 (+ 240us) server_negotiation.cc:407] Connection header received
0806 16:03:17.538377 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:17.538380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:17.538423 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:17.538511 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:17.539166 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:17.539911 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:17.541063 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:17.541383 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:17.541460 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:17.543200 (+ 1740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:17.543204 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:17.543207 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:17.543231 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:17.544743 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:17.545173 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:17.545177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:17.545179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:17.545260 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:17.545540 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:17.545545 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:17.545549 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:17.545719 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:17.545852 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:17.546102 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 16:03:17.546222 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 16:03:17.582427 14637 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:17.583603 14637 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:17.593278 14637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62cfdc30084c480f97d23c785477597b: Generated new TSK 0
I20260806 16:03:17.594007 14637 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:17.598294 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 14553
2026-08-06T16:03:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1448 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:17 dist-test-slave-tm1g krb5kdc[14655](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:17 dist-test-slave-tm1g krb5kdc[14655](info): set up 2 sockets
Aug 06 16:03:17 dist-test-slave-tm1g krb5kdc[14655](info): commencing operation
krb5kdc: starting...
W20260806 16:03:18.647154 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:18 dist-test-slave-tm1g krb5kdc[14655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:03:18.763294 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:18.873052 14668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:18.873319 14668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:18.873368 14668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:18.877342 14668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:18.877398 14668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:18.877417 14668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:18.877434 14668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:18.877450 14668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:18.882663 14668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:18.883657 14668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:18.884649 14668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:18.893154 14673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:18.893152 14674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:18.893187 14676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:18.893491 14668 server_base.cc:1034] running on GCE node
I20260806 16:03:18.894093 14668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:18.895184 14668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:18.896375 14668 hybrid_clock.cc:648] HybridClock initialized: now 1786032198896343 us; error 42 us; skew 500 ppm
Aug 06 16:03:18 dist-test-slave-tm1g krb5kdc[14655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032198, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:18.899151 14668 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:03:18.900058 14668 webserver.cc:460] Webserver started at http://127.0.0.1:39577/ using document root <none> and password file <none>
I20260806 16:03:18.900611 14668 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:18.900704 14668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:18.901067 14668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:18.903201 14668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "1db97aef35eb4903b0edc4df69d9c641"
format_stamp: "Formatted at 2026-08-06 16:03:18 on dist-test-slave-tm1g"
server_key: "90247538fba855ae770fed40aea58073"
server_key_iv: "a3d2aea29baee49528d14966e3e984bd"
server_key_version: "encryptionkey@0"
I20260806 16:03:18.903774 14668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "1db97aef35eb4903b0edc4df69d9c641"
format_stamp: "Formatted at 2026-08-06 16:03:18 on dist-test-slave-tm1g"
server_key: "90247538fba855ae770fed40aea58073"
server_key_iv: "a3d2aea29baee49528d14966e3e984bd"
server_key_version: "encryptionkey@0"
I20260806 16:03:18.907478 14668 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:03:18.909843 14683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:18.911147 14668 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 16:03:18.911288 14668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "1db97aef35eb4903b0edc4df69d9c641"
format_stamp: "Formatted at 2026-08-06 16:03:18 on dist-test-slave-tm1g"
server_key: "90247538fba855ae770fed40aea58073"
server_key_iv: "a3d2aea29baee49528d14966e3e984bd"
server_key_version: "encryptionkey@0"
I20260806 16:03:18.911397 14668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:18.921078 14668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:18.924062 14668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:18.924261 14668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:18.931362 14668 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46773
I20260806 16:03:18.931396 14737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46773 every 8 connection(s)
I20260806 16:03:18.932271 14668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:18.935035 14738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:18.939173 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 14668
I20260806 16:03:18.939317 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:18.939599 25726 external_mini_cluster.cc:1428] Setting key ba0e5f12d1827f845d25c76a848faa59
I20260806 16:03:18.940876 14738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641: Bootstrap starting.
I20260806 16:03:18.943593 14738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:18.944473 14738 log.cc:824] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:18.946527 14738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641: No bootstrap required, opened a new log
I20260806 16:03:18.949641 14741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:18.941047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59391 (local address 127.0.0.1:46773)
0806 16:03:18.941480 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:18.941499 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:18.941541 (+ 42us) server_negotiation.cc:407] Connection header received
0806 16:03:18.942403 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:18.942426 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:18.942838 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:18.943192 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:18.943637 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:18.945051 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:18.945979 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:18.946685 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:18.946850 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:18.947636 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:18.947642 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:18.947657 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:18.948157 (+ 500us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:18.948219 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:18.948247 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:18.948358 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:18.948462 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 16:03:18.948670 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":122,"threads_started":1}
I20260806 16:03:18.951090 14738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db97aef35eb4903b0edc4df69d9c641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46773 } }
I20260806 16:03:18.951299 14738 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:18.951361 14738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1db97aef35eb4903b0edc4df69d9c641, State: Initialized, Role: FOLLOWER
I20260806 16:03:18.951814 14738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db97aef35eb4903b0edc4df69d9c641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46773 } }
I20260806 16:03:18.951982 14738 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:18.952057 14738 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:18.952178 14738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:18.952553 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:18.953164 14738 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db97aef35eb4903b0edc4df69d9c641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46773 } }
I20260806 16:03:18.953495 14738 leader_election.cc:302] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1db97aef35eb4903b0edc4df69d9c641; no voters:
I20260806 16:03:18.953819 14738 leader_election.cc:288] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:18.953899 14745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:18.954090 14745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [term 1 LEADER]: Becoming Leader. State: Replica: 1db97aef35eb4903b0edc4df69d9c641, State: Running, Role: LEADER
I20260806 16:03:18.954464 14745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db97aef35eb4903b0edc4df69d9c641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46773 } }
I20260806 16:03:18.955217 14738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:18.955667 14749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1db97aef35eb4903b0edc4df69d9c641. Latest consensus state: current_term: 1 leader_uuid: "1db97aef35eb4903b0edc4df69d9c641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db97aef35eb4903b0edc4df69d9c641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46773 } } }
I20260806 16:03:18.955790 14749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:18.956431 14748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1db97aef35eb4903b0edc4df69d9c641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1db97aef35eb4903b0edc4df69d9c641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46773 } } }
I20260806 16:03:18.956606 14748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641 [sys.catalog]: This master's current role is: LEADER
W20260806 16:03:18.960379 14765 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 16:03:18.960423 14741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:18.954610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43969 (local address 127.0.0.1:46773)
0806 16:03:18.954713 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:18.954717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:18.955826 (+ 1109us) server_negotiation.cc:407] Connection header received
0806 16:03:18.956025 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:18.956031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:18.956086 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:18.956145 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:18.956616 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:18.957380 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:18.958579 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:18.958913 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:18.959030 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:18.960012 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:18.960015 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:18.960018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:18.960030 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:18.960058 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:18.960072 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:18.960149 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:18.960248 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 16:03:18.960300 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 16:03:18.960661 14765 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:03:18.960760 14753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:18.961412 14753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:18.966662 14753 catalog_manager.cc:1347] Generated new cluster ID: 0487b5076377434d9a8263e1e4cec440
I20260806 16:03:18.966728 14753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:18.981353 14753 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:18.982427 14753 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:18.986173 14753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1db97aef35eb4903b0edc4df69d9c641: Generated new TSK 0
I20260806 16:03:18.986819 14753 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:18.998623 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 14668
2026-08-06T16:03:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1401 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:19 dist-test-slave-tm1g krb5kdc[14770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:19 dist-test-slave-tm1g krb5kdc[14770](info): set up 2 sockets
Aug 06 16:03:19 dist-test-slave-tm1g krb5kdc[14770](info): commencing operation
krb5kdc: starting...
W20260806 16:03:20.074910 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:20 dist-test-slave-tm1g krb5kdc[14770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:03:20.195361 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37001
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:20.305711 14783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:20.305948 14783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:20.305989 14783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:20.309967 14783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:20.310024 14783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:20.310050 14783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:20.310096 14783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:20.310130 14783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:20.315310 14783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:20.316314 14783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:20.317303 14783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:20.325943 14789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:20.325994 14791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:20.326116 14788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:20.326145 14783 server_base.cc:1034] running on GCE node
I20260806 16:03:20.326797 14783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:20.327787 14783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:20.328950 14783 hybrid_clock.cc:648] HybridClock initialized: now 1786032200328922 us; error 41 us; skew 500 ppm
Aug 06 16:03:20 dist-test-slave-tm1g krb5kdc[14770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032200, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:20.331815 14783 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:03:20.332722 14783 webserver.cc:460] Webserver started at http://127.0.0.1:38417/ using document root <none> and password file <none>
I20260806 16:03:20.333281 14783 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:20.333343 14783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:20.333588 14783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:20.335654 14783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "cc70810a773842cabd7629e9772c9406"
format_stamp: "Formatted at 2026-08-06 16:03:20 on dist-test-slave-tm1g"
server_key: "5de7b48801553028fe5777fea1631c58"
server_key_iv: "f9794d93ddd83666afb5acfdf9774550"
server_key_version: "encryptionkey@0"
I20260806 16:03:20.336223 14783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "cc70810a773842cabd7629e9772c9406"
format_stamp: "Formatted at 2026-08-06 16:03:20 on dist-test-slave-tm1g"
server_key: "5de7b48801553028fe5777fea1631c58"
server_key_iv: "f9794d93ddd83666afb5acfdf9774550"
server_key_version: "encryptionkey@0"
I20260806 16:03:20.339869 14783 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 16:03:20.342618 14798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:20.343863 14783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:03:20.344023 14783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "cc70810a773842cabd7629e9772c9406"
format_stamp: "Formatted at 2026-08-06 16:03:20 on dist-test-slave-tm1g"
server_key: "5de7b48801553028fe5777fea1631c58"
server_key_iv: "f9794d93ddd83666afb5acfdf9774550"
server_key_version: "encryptionkey@0"
I20260806 16:03:20.344146 14783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:20.348788 14783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:20.351734 14783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:20.351930 14783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:20.359138 14783 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37001
I20260806 16:03:20.359158 14850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37001 every 8 connection(s)
I20260806 16:03:20.360028 14783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:20.361236 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 14783
I20260806 16:03:20.361387 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:20.361685 25726 external_mini_cluster.cc:1428] Setting key 77cd9ea22b7f1a02d47d5dd48b493672
I20260806 16:03:20.363099 14851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:20.366685 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:20.363004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40529 (local address 127.0.0.1:37001)
0806 16:03:20.363421 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:20.363438 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:20.363474 (+ 36us) server_negotiation.cc:407] Connection header received
0806 16:03:20.364095 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:20.364110 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:20.364376 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:20.364653 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:20.365443 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:20.365448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:20.365458 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:20.365468 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:20.365517 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:20.365534 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:20.365643 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:20.365749 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 16:03:20.365910 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":141,"threads_started":1}
I20260806 16:03:20.369473 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:20.369510 14851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406: Bootstrap starting.
I20260806 16:03:20.372191 14851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:20.373050 14851 log.cc:824] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:20.373479 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:20.371761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34239 (local address 127.0.0.1:37001)
0806 16:03:20.371900 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:20.371904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:20.372089 (+ 185us) server_negotiation.cc:407] Connection header received
0806 16:03:20.372220 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:20.372222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:20.372269 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:20.372377 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:20.373085 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:20.373088 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:20.373090 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:20.373102 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:20.373134 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:20.373149 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:20.373227 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:20.373301 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 16:03:20.373339 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 16:03:20.375079 14851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406: No bootstrap required, opened a new log
I20260806 16:03:20.378490 14851 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc70810a773842cabd7629e9772c9406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37001 } }
I20260806 16:03:20.378772 14851 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:20.378844 14851 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc70810a773842cabd7629e9772c9406, State: Initialized, Role: FOLLOWER
I20260806 16:03:20.379246 14851 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc70810a773842cabd7629e9772c9406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37001 } }
I20260806 16:03:20.379397 14851 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:20.379534 14851 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:20.379676 14851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:20.380697 14851 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc70810a773842cabd7629e9772c9406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37001 } }
I20260806 16:03:20.381031 14851 leader_election.cc:302] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc70810a773842cabd7629e9772c9406; no voters:
I20260806 16:03:20.381381 14851 leader_election.cc:288] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:20.381484 14861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:20.381734 14861 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [term 1 LEADER]: Becoming Leader. State: Replica: cc70810a773842cabd7629e9772c9406, State: Running, Role: LEADER
I20260806 16:03:20.382059 14861 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc70810a773842cabd7629e9772c9406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37001 } }
I20260806 16:03:20.382649 14851 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:20.383721 14863 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc70810a773842cabd7629e9772c9406. Latest consensus state: current_term: 1 leader_uuid: "cc70810a773842cabd7629e9772c9406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc70810a773842cabd7629e9772c9406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37001 } } }
I20260806 16:03:20.383850 14863 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:20.384229 14871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:20.384182 14862 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc70810a773842cabd7629e9772c9406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc70810a773842cabd7629e9772c9406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37001 } } }
I20260806 16:03:20.384409 14862 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:20.387173 14871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:20.392510 14871 catalog_manager.cc:1347] Generated new cluster ID: 5c4712393f6d447ca4f37ca1571caaad
I20260806 16:03:20.392572 14871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:20.407938 14871 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:20.408332 14871 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:20.414976 14871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc70810a773842cabd7629e9772c9406: Generated new TSK 0
I20260806 16:03:20.415817 14871 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:20.421842 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 14783
2026-08-06T16:03:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:20 dist-test-slave-tm1g krb5kdc[14883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:20 dist-test-slave-tm1g krb5kdc[14883](info): set up 2 sockets
Aug 06 16:03:20 dist-test-slave-tm1g krb5kdc[14883](info): commencing operation
krb5kdc: starting...
W20260806 16:03:21.470857 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:21 dist-test-slave-tm1g krb5kdc[14883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:03:21.587721 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40075
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:21.696538 14896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:21.696779 14896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:21.696818 14896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:21.700621 14896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:21.700666 14896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:21.700685 14896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:21.700700 14896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:21.700716 14896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:21.705813 14896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:21.706763 14896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:21.707701 14896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:21.716192 14901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:21.716243 14902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:21.716298 14896 server_base.cc:1034] running on GCE node
W20260806 16:03:21.716192 14904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:21.717118 14896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:21.718104 14896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:21.719262 14896 hybrid_clock.cc:648] HybridClock initialized: now 1786032201719226 us; error 49 us; skew 500 ppm
Aug 06 16:03:21 dist-test-slave-tm1g krb5kdc[14883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:21.726711 14896 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:03:21.727620 14896 webserver.cc:460] Webserver started at http://127.0.0.1:35805/ using document root <none> and password file <none>
I20260806 16:03:21.728184 14896 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:21.728276 14896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:21.728508 14896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:21.730621 14896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "253ce2b22e0c4af0bc69587972b45230"
format_stamp: "Formatted at 2026-08-06 16:03:21 on dist-test-slave-tm1g"
server_key: "485248b966130054da54cf166baa53de"
server_key_iv: "29f15297bc1d8946591bad9e25a27a94"
server_key_version: "encryptionkey@0"
I20260806 16:03:21.731416 14896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "253ce2b22e0c4af0bc69587972b45230"
format_stamp: "Formatted at 2026-08-06 16:03:21 on dist-test-slave-tm1g"
server_key: "485248b966130054da54cf166baa53de"
server_key_iv: "29f15297bc1d8946591bad9e25a27a94"
server_key_version: "encryptionkey@0"
I20260806 16:03:21.735217 14896 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:03:21.737589 14911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:21.738785 14896 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:03:21.738956 14896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "253ce2b22e0c4af0bc69587972b45230"
format_stamp: "Formatted at 2026-08-06 16:03:21 on dist-test-slave-tm1g"
server_key: "485248b966130054da54cf166baa53de"
server_key_iv: "29f15297bc1d8946591bad9e25a27a94"
server_key_version: "encryptionkey@0"
I20260806 16:03:21.739097 14896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:21.756608 14896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:21.759442 14896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:21.759646 14896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:21.766806 14896 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40075
I20260806 16:03:21.766824 14963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40075 every 8 connection(s)
I20260806 16:03:21.767707 14896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:21.770372 14964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:21.773245 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 14896
I20260806 16:03:21.773401 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:21.773686 25726 external_mini_cluster.cc:1428] Setting key 627862934c392a7ef07ee53c418079f4
I20260806 16:03:21.776180 14964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230: Bootstrap starting.
I20260806 16:03:21.778791 14964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:21.779633 14964 log.cc:824] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:21.781757 14964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230: No bootstrap required, opened a new log
Aug 06 16:03:21 dist-test-slave-tm1g krb5kdc[14883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 16:03:21.785233 14964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253ce2b22e0c4af0bc69587972b45230" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40075 } }
I20260806 16:03:21.785460 14964 raft_consensus.cc:374] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:21.785616 14964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 253ce2b22e0c4af0bc69587972b45230, State: Initialized, Role: FOLLOWER
I20260806 16:03:21.786120 14964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253ce2b22e0c4af0bc69587972b45230" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40075 } }
I20260806 16:03:21.786319 14964 raft_consensus.cc:388] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:21.786408 14964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:21.786518 14964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:21.787539 14964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253ce2b22e0c4af0bc69587972b45230" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40075 } }
I20260806 16:03:21.787915 14964 leader_election.cc:302] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 253ce2b22e0c4af0bc69587972b45230; no voters:
I20260806 16:03:21.788247 14964 leader_election.cc:288] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:21.788379 14969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:21.788652 14969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [term 1 LEADER]: Becoming Leader. State: Replica: 253ce2b22e0c4af0bc69587972b45230, State: Running, Role: LEADER
I20260806 16:03:21.789021 14969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253ce2b22e0c4af0bc69587972b45230" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40075 } }
I20260806 16:03:21.789533 14964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:21.790225 14967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:21.775092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:44903 (local address 127.0.0.1:40075)
0806 16:03:21.775488 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:21.775507 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:21.775550 (+ 43us) server_negotiation.cc:407] Connection header received
0806 16:03:21.776351 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:21.776372 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:21.776782 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:21.777147 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:21.777797 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:21.779315 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:21.780395 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:21.781090 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:21.781247 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:21.784073 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:21.784078 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:21.784091 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:21.784116 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:21.786661 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:21.787149 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:21.787155 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:21.787161 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:21.787306 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:21.787634 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:21.787640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:21.787644 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:21.788147 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:21.788380 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:21.789036 (+ 656us) server_negotiation.cc:299] Negotiation successful
0806 16:03:21.789262 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":104,"threads_started":1}
I20260806 16:03:21.790756 14970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "253ce2b22e0c4af0bc69587972b45230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253ce2b22e0c4af0bc69587972b45230" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40075 } } }
I20260806 16:03:21.790874 14970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:21.791182 14971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 253ce2b22e0c4af0bc69587972b45230. Latest consensus state: current_term: 1 leader_uuid: "253ce2b22e0c4af0bc69587972b45230" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "253ce2b22e0c4af0bc69587972b45230" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40075 } } }
I20260806 16:03:21.791271 14971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:21.793617 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 16:03:21.794911 14984 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:03:21.794991 14984 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:03:21.795148 14978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:21.796010 14978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:21.801815 14978 catalog_manager.cc:1347] Generated new cluster ID: 988009270a45446386c776305b2ea4ee
I20260806 16:03:21.801882 14978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:21.806984 14967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:21.798195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:57567 (local address 127.0.0.1:40075)
0806 16:03:21.798292 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:21.798297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:21.798546 (+ 249us) server_negotiation.cc:407] Connection header received
0806 16:03:21.798648 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:21.798651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:21.798697 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:21.798790 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:21.799576 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:21.800682 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:21.801785 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:21.802152 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:21.802268 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:21.803661 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:21.803665 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:21.803668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:21.803691 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:21.805146 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:21.805640 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:21.805643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:21.805645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:21.805728 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:21.806081 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:21.806085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:21.806086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:21.806259 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:21.806449 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:21.806725 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 16:03:21.806817 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 16:03:21.824266 14978 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:21.825151 14978 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:21.839622 14978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 253ce2b22e0c4af0bc69587972b45230: Generated new TSK 0
I20260806 16:03:21.840370 14978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:21.851987 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 14896
W20260806 16:03:21.858287 14881 connection.cc:535] client connection to 127.0.0.1:40075 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40075): Connection reset by peer (error 104)
2026-08-06T16:03:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:21 dist-test-slave-tm1g krb5kdc[14995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:21 dist-test-slave-tm1g krb5kdc[14995](info): set up 2 sockets
Aug 06 16:03:21 dist-test-slave-tm1g krb5kdc[14995](info): commencing operation
krb5kdc: starting...
W20260806 16:03:22.907296 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:22 dist-test-slave-tm1g krb5kdc[14995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:03:23.029050 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42629
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:23.136821 15008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:23.137123 15008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:23.137199 15008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:23.141136 15008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:23.141229 15008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:23.141269 15008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:23.141330 15008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:23.141368 15008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:23.146606 15008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:23.147632 15008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:23.148626 15008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:23.157045 15014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:23.157035 15017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:23.157035 15015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:23.157302 15008 server_base.cc:1034] running on GCE node
I20260806 16:03:23.158041 15008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:23.159101 15008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:23.160266 15008 hybrid_clock.cc:648] HybridClock initialized: now 1786032203160243 us; error 42 us; skew 500 ppm
Aug 06 16:03:23 dist-test-slave-tm1g krb5kdc[14995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032203, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:23.163049 15008 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:03:23.163950 15008 webserver.cc:460] Webserver started at http://127.0.0.1:37017/ using document root <none> and password file <none>
I20260806 16:03:23.164507 15008 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:23.164599 15008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:23.164834 15008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:23.166898 15008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "8bf71de31a9d48a583d392f8d1910a9a"
format_stamp: "Formatted at 2026-08-06 16:03:23 on dist-test-slave-tm1g"
server_key: "8a79bad98a3a0143c9beacc2dbd8f670"
server_key_iv: "5ecbbb982a7ce392a6cbbc3fd4720300"
server_key_version: "encryptionkey@0"
I20260806 16:03:23.167464 15008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "8bf71de31a9d48a583d392f8d1910a9a"
format_stamp: "Formatted at 2026-08-06 16:03:23 on dist-test-slave-tm1g"
server_key: "8a79bad98a3a0143c9beacc2dbd8f670"
server_key_iv: "5ecbbb982a7ce392a6cbbc3fd4720300"
server_key_version: "encryptionkey@0"
I20260806 16:03:23.171051 15008 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:03:23.173369 15024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:23.174652 15008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:03:23.174790 15008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "8bf71de31a9d48a583d392f8d1910a9a"
format_stamp: "Formatted at 2026-08-06 16:03:23 on dist-test-slave-tm1g"
server_key: "8a79bad98a3a0143c9beacc2dbd8f670"
server_key_iv: "5ecbbb982a7ce392a6cbbc3fd4720300"
server_key_version: "encryptionkey@0"
I20260806 16:03:23.174901 15008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:23.197309 15008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:23.200662 15008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:23.200872 15008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:23.208199 15008 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42629
I20260806 16:03:23.208220 15076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42629 every 8 connection(s)
I20260806 16:03:23.209149 15008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:23.211901 15077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:23.215236 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 15008
I20260806 16:03:23.215384 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:23.215668 25726 external_mini_cluster.cc:1428] Setting key a05390f3a0102b69e39486e8f1f2dc5a
I20260806 16:03:23.217900 15077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a: Bootstrap starting.
I20260806 16:03:23.220075 15077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:23.220810 15077 log.cc:824] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:23.222656 15077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a: No bootstrap required, opened a new log
I20260806 16:03:23.224680 15080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:23.217103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:44139 (local address 127.0.0.1:42629)
0806 16:03:23.217549 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:23.217564 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:23.217599 (+ 35us) server_negotiation.cc:407] Connection header received
0806 16:03:23.218267 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:23.218284 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:23.218681 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:23.218995 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:23.219393 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:23.220410 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:23.221333 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:23.221800 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:23.221972 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:23.222914 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:23.222919 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:23.222930 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:23.223290 (+ 360us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:23.223340 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:23.223362 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:23.223510 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:23.223683 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 16:03:23.223817 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":133,"threads_started":1}
I20260806 16:03:23.225776 15077 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf71de31a9d48a583d392f8d1910a9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } }
I20260806 16:03:23.226032 15077 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:23.226092 15077 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf71de31a9d48a583d392f8d1910a9a, State: Initialized, Role: FOLLOWER
I20260806 16:03:23.226557 15077 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf71de31a9d48a583d392f8d1910a9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } }
I20260806 16:03:23.226720 15077 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:23.226794 15077 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:23.226897 15077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:23.227545 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:23.227916 15077 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf71de31a9d48a583d392f8d1910a9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } }
I20260806 16:03:23.228248 15077 leader_election.cc:302] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bf71de31a9d48a583d392f8d1910a9a; no voters:
I20260806 16:03:23.228570 15077 leader_election.cc:288] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:23.228639 15082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:23.228868 15082 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [term 1 LEADER]: Becoming Leader. State: Replica: 8bf71de31a9d48a583d392f8d1910a9a, State: Running, Role: LEADER
I20260806 16:03:23.229230 15082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf71de31a9d48a583d392f8d1910a9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } }
I20260806 16:03:23.229804 15077 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:23.230398 15082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bf71de31a9d48a583d392f8d1910a9a. Latest consensus state: current_term: 1 leader_uuid: "8bf71de31a9d48a583d392f8d1910a9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf71de31a9d48a583d392f8d1910a9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } } }
I20260806 16:03:23.230535 15082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:23.230937 15086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bf71de31a9d48a583d392f8d1910a9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf71de31a9d48a583d392f8d1910a9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42629 } } }
I20260806 16:03:23.231088 15086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:23.232187 15092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:23.235119 15092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:23.235657 15080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:23.231249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:57243 (local address 127.0.0.1:42629)
0806 16:03:23.231344 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:23.231348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:23.231784 (+ 436us) server_negotiation.cc:407] Connection header received
0806 16:03:23.231956 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:23.231959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:23.232013 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:23.232100 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:23.232527 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:23.233092 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:23.234004 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:23.234234 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:23.234372 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:23.235189 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:23.235194 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:23.235200 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:23.235214 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:23.235242 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:23.235255 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:23.235360 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:23.235483 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 16:03:23.235512 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 16:03:23.240304 15092 catalog_manager.cc:1347] Generated new cluster ID: 131a3b7ae5714af999cc04a2da6e679b
I20260806 16:03:23.240373 15092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:23.250679 15092 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:23.251838 15092 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:23.260476 15092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bf71de31a9d48a583d392f8d1910a9a: Generated new TSK 0
I20260806 16:03:23.261227 15092 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:23.265103 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 15008
W20260806 16:03:23.271246 14993 connection.cc:535] client connection to 127.0.0.1:42629 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42629): Connection reset by peer (error 104)
2026-08-06T16:03:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:23 dist-test-slave-tm1g krb5kdc[15108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:23 dist-test-slave-tm1g krb5kdc[15108](info): set up 2 sockets
Aug 06 16:03:23 dist-test-slave-tm1g krb5kdc[15108](info): commencing operation
krb5kdc: starting...
W20260806 16:03:24.310783 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:24 dist-test-slave-tm1g krb5kdc[15108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:03:24.428716 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38751
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:24.535013 15121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:24.535244 15121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:24.535322 15121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:24.539218 15121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:24.539266 15121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:24.539285 15121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:24.539359 15121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:24.539398 15121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:24.545553 15121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:24.546581 15121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:24.547549 15121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:24.555666 15129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:24.555691 15127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:24.555946 15126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:24.556068 15121 server_base.cc:1034] running on GCE node
I20260806 16:03:24.556571 15121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:24.557572 15121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:24.558728 15121 hybrid_clock.cc:648] HybridClock initialized: now 1786032204558711 us; error 29 us; skew 500 ppm
Aug 06 16:03:24 dist-test-slave-tm1g krb5kdc[15108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032204, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:24.561450 15121 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:03:24.562357 15121 webserver.cc:460] Webserver started at http://127.0.0.1:41649/ using document root <none> and password file <none>
I20260806 16:03:24.562891 15121 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:24.562951 15121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:24.563198 15121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:24.565194 15121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "cd28c63d350c4ca28d3c4043e2f8ba49"
format_stamp: "Formatted at 2026-08-06 16:03:24 on dist-test-slave-tm1g"
server_key: "df37677e268266af20fd95fef0463b2c"
server_key_iv: "be5c967de645ad8b7f74a6b2bdde7bd6"
server_key_version: "encryptionkey@0"
I20260806 16:03:24.565778 15121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "cd28c63d350c4ca28d3c4043e2f8ba49"
format_stamp: "Formatted at 2026-08-06 16:03:24 on dist-test-slave-tm1g"
server_key: "df37677e268266af20fd95fef0463b2c"
server_key_iv: "be5c967de645ad8b7f74a6b2bdde7bd6"
server_key_version: "encryptionkey@0"
I20260806 16:03:24.569342 15121 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 16:03:24.571662 15136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:24.572877 15121 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:03:24.573028 15121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "cd28c63d350c4ca28d3c4043e2f8ba49"
format_stamp: "Formatted at 2026-08-06 16:03:24 on dist-test-slave-tm1g"
server_key: "df37677e268266af20fd95fef0463b2c"
server_key_iv: "be5c967de645ad8b7f74a6b2bdde7bd6"
server_key_version: "encryptionkey@0"
I20260806 16:03:24.573135 15121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:24.577616 15121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:24.580721 15121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:24.580921 15121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:24.588042 15121 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38751
I20260806 16:03:24.588073 15189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38751 every 8 connection(s)
I20260806 16:03:24.588901 15121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:24.591454 15190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:24.593958 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 15121
I20260806 16:03:24.594102 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:24.594444 25726 external_mini_cluster.cc:1428] Setting key f51d4d540ca84c850ad7bfd4da6c1106
I20260806 16:03:24.597421 15190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49: Bootstrap starting.
I20260806 16:03:24.599756 15193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:24.595780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:33673 (local address 127.0.0.1:38751)
0806 16:03:24.596200 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:24.596220 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:24.596262 (+ 42us) server_negotiation.cc:407] Connection header received
0806 16:03:24.597033 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:24.597055 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:24.597382 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:24.597740 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:24.598423 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:24.598429 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:24.598442 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:24.598454 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:24.598528 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:24.598558 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:24.598708 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:24.598781 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 16:03:24.598927 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":109,"threads_started":1}
I20260806 16:03:24.600090 15190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:24.600800 15190 log.cc:824] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:24.602439 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:24.603022 15190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49: No bootstrap required, opened a new log
I20260806 16:03:24.606031 15193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:24.604490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:36119 (local address 127.0.0.1:38751)
0806 16:03:24.604612 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:24.604616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:24.604813 (+ 197us) server_negotiation.cc:407] Connection header received
0806 16:03:24.604941 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:24.604943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:24.604968 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:24.605031 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:24.605665 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:24.605667 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:24.605669 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:24.605679 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:24.605702 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:24.605712 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:24.605778 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:24.605886 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 16:03:24.605913 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 16:03:24.607190 15190 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd28c63d350c4ca28d3c4043e2f8ba49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38751 } }
I20260806 16:03:24.607435 15190 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:24.607492 15190 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd28c63d350c4ca28d3c4043e2f8ba49, State: Initialized, Role: FOLLOWER
I20260806 16:03:24.607864 15190 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd28c63d350c4ca28d3c4043e2f8ba49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38751 } }
I20260806 16:03:24.608007 15190 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:24.608070 15190 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:24.608161 15190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:24.608984 15190 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd28c63d350c4ca28d3c4043e2f8ba49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38751 } }
I20260806 16:03:24.609300 15190 leader_election.cc:302] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd28c63d350c4ca28d3c4043e2f8ba49; no voters:
I20260806 16:03:24.609606 15190 leader_election.cc:288] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:24.609709 15200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:24.609954 15200 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [term 1 LEADER]: Becoming Leader. State: Replica: cd28c63d350c4ca28d3c4043e2f8ba49, State: Running, Role: LEADER
I20260806 16:03:24.610280 15200 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd28c63d350c4ca28d3c4043e2f8ba49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38751 } }
I20260806 16:03:24.610826 15190 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:24.611920 15202 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd28c63d350c4ca28d3c4043e2f8ba49. Latest consensus state: current_term: 1 leader_uuid: "cd28c63d350c4ca28d3c4043e2f8ba49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd28c63d350c4ca28d3c4043e2f8ba49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38751 } } }
I20260806 16:03:24.612048 15202 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:24.612425 15209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:24.612406 15201 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd28c63d350c4ca28d3c4043e2f8ba49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd28c63d350c4ca28d3c4043e2f8ba49" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38751 } } }
I20260806 16:03:24.613040 15201 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:24.615512 15209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:24.620368 15209 catalog_manager.cc:1347] Generated new cluster ID: f070242642cf40f2bfbc95024dc666d4
I20260806 16:03:24.620428 15209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:24.627319 15209 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:24.627604 15209 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:24.635066 15209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd28c63d350c4ca28d3c4043e2f8ba49: Generated new TSK 0
I20260806 16:03:24.635646 15209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:24.641335 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 15121
2026-08-06T16:03:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1375 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:24 dist-test-slave-tm1g krb5kdc[15221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:24 dist-test-slave-tm1g krb5kdc[15221](info): set up 2 sockets
Aug 06 16:03:24 dist-test-slave-tm1g krb5kdc[15221](info): commencing operation
krb5kdc: starting...
W20260806 16:03:25.690928 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:25 dist-test-slave-tm1g krb5kdc[15221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:03:25.813477 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46283
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:25.921056 15234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:25.921324 15234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:25.921402 15234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:25.925375 15234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:25.925467 15234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:25.925506 15234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:25.925563 15234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:25.925601 15234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:25.930660 15234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46283
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:25.931630 15234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:25.932596 15234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:25.941262 15239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:25.941269 15242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:25.941269 15240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:25.941491 15234 server_base.cc:1034] running on GCE node
I20260806 16:03:25.942127 15234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:25.943167 15234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:25.944329 15234 hybrid_clock.cc:648] HybridClock initialized: now 1786032205944312 us; error 31 us; skew 500 ppm
Aug 06 16:03:25 dist-test-slave-tm1g krb5kdc[15221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032205, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:25.946914 15234 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:03:25.947821 15234 webserver.cc:460] Webserver started at http://127.0.0.1:35915/ using document root <none> and password file <none>
I20260806 16:03:25.948350 15234 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:25.948447 15234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:25.948670 15234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:25.950675 15234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "175dfb43d32d439d82e7d37cd9692763"
format_stamp: "Formatted at 2026-08-06 16:03:25 on dist-test-slave-tm1g"
server_key: "32a5c2050aae38a714635900a22ad997"
server_key_iv: "1ad7d57a704e0b8763f4b65e9bd3a57b"
server_key_version: "encryptionkey@0"
I20260806 16:03:25.951233 15234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "175dfb43d32d439d82e7d37cd9692763"
format_stamp: "Formatted at 2026-08-06 16:03:25 on dist-test-slave-tm1g"
server_key: "32a5c2050aae38a714635900a22ad997"
server_key_iv: "1ad7d57a704e0b8763f4b65e9bd3a57b"
server_key_version: "encryptionkey@0"
I20260806 16:03:25.954777 15234 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:03:25.957067 15249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:25.958250 15234 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:03:25.958425 15234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "175dfb43d32d439d82e7d37cd9692763"
format_stamp: "Formatted at 2026-08-06 16:03:25 on dist-test-slave-tm1g"
server_key: "32a5c2050aae38a714635900a22ad997"
server_key_iv: "1ad7d57a704e0b8763f4b65e9bd3a57b"
server_key_version: "encryptionkey@0"
I20260806 16:03:25.958552 15234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:25.985419 15234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:25.988521 15234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:25.988735 15234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:25.995990 15234 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46283
I20260806 16:03:25.996026 15302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46283 every 8 connection(s)
I20260806 16:03:25.996939 15234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:25.999142 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 15234
I20260806 16:03:25.999287 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:25.999584 25726 external_mini_cluster.cc:1428] Setting key 188fe82f2084128d3e49732a8800f3bd
I20260806 16:03:25.999742 15303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:26.007071 15303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763: Bootstrap starting.
Aug 06 16:03:26 dist-test-slave-tm1g krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 16:03:26.009414 15303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:26.010335 15303 log.cc:824] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:26.012162 15303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763: No bootstrap required, opened a new log
I20260806 16:03:26.014734 15306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:26.001021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:55383 (local address 127.0.0.1:46283)
0806 16:03:26.001439 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:26.001457 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:26.001499 (+ 42us) server_negotiation.cc:407] Connection header received
0806 16:03:26.002531 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:26.002555 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:26.002993 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:26.003353 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:26.004009 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:26.005514 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:26.006424 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:26.007118 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:26.007290 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:26.009823 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:26.009829 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:26.009845 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:26.009882 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:26.011730 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:26.012268 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:26.012276 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:26.012284 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:26.012412 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:26.012769 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:26.012773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:26.012774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:26.013167 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:26.013397 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:26.013746 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 16:03:26.013964 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":120,"threads_started":1}
I20260806 16:03:26.015897 15303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175dfb43d32d439d82e7d37cd9692763" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46283 } }
I20260806 16:03:26.016160 15303 raft_consensus.cc:374] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:26.016220 15303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 175dfb43d32d439d82e7d37cd9692763, State: Initialized, Role: FOLLOWER
I20260806 16:03:26.016680 15303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175dfb43d32d439d82e7d37cd9692763" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46283 } }
I20260806 16:03:26.016810 15303 raft_consensus.cc:388] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:26.016896 15303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:26.017024 15303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:26.017684 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:26.018093 15303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175dfb43d32d439d82e7d37cd9692763" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46283 } }
I20260806 16:03:26.018481 15303 leader_election.cc:302] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 175dfb43d32d439d82e7d37cd9692763; no voters:
I20260806 16:03:26.018854 15303 leader_election.cc:288] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:26.018922 15308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:26.019125 15308 raft_consensus.cc:686] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [term 1 LEADER]: Becoming Leader. State: Replica: 175dfb43d32d439d82e7d37cd9692763, State: Running, Role: LEADER
I20260806 16:03:26.019414 15308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175dfb43d32d439d82e7d37cd9692763" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46283 } }
I20260806 16:03:26.020128 15303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:26.020571 15308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 175dfb43d32d439d82e7d37cd9692763. Latest consensus state: current_term: 1 leader_uuid: "175dfb43d32d439d82e7d37cd9692763" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175dfb43d32d439d82e7d37cd9692763" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46283 } } }
I20260806 16:03:26.020735 15308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:26.020989 15308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "175dfb43d32d439d82e7d37cd9692763" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "175dfb43d32d439d82e7d37cd9692763" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46283 } } }
I20260806 16:03:26.021144 15308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:26.021461 15318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:26.024734 15318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:26.030361 15318 catalog_manager.cc:1347] Generated new cluster ID: 38e3b63697a446ceb5fe0ae24ea6d20c
I20260806 16:03:26.030424 15318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:26.030792 15306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:26.019841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:50689 (local address 127.0.0.1:46283)
0806 16:03:26.019952 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:26.019956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:26.020202 (+ 246us) server_negotiation.cc:407] Connection header received
0806 16:03:26.020351 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:26.020354 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:26.020400 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:26.020470 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:26.021428 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:26.022200 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:26.023433 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:26.023681 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:26.023794 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:26.025650 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:26.025654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:26.025656 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:26.025680 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:26.028790 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:26.029303 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:26.029307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:26.029309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:26.029397 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:26.029739 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:26.029743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:26.029745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:26.029924 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:26.030087 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:26.030550 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 16:03:26.030652 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 16:03:26.042537 15318 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:26.043416 15318 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:26.049700 15318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 175dfb43d32d439d82e7d37cd9692763: Generated new TSK 0
I20260806 16:03:26.050518 15318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:26.056790 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 15234
W20260806 16:03:26.063207 15219 connection.cc:535] client connection to 127.0.0.1:46283 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46283): Connection reset by peer (error 104)
2026-08-06T16:03:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1415 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:26 dist-test-slave-tm1g krb5kdc[15334](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:26 dist-test-slave-tm1g krb5kdc[15334](info): set up 2 sockets
Aug 06 16:03:26 dist-test-slave-tm1g krb5kdc[15334](info): commencing operation
krb5kdc: starting...
W20260806 16:03:27.111054 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:27 dist-test-slave-tm1g krb5kdc[15334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:03:27.229238 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35767
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35767
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:27.338045 15347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:27.338284 15347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:27.338425 15347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:27.342347 15347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:27.342396 15347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:27.342413 15347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:27.342465 15347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:27.342494 15347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:27.347705 15347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35767
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:27.348666 15347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:27.349635 15347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:27.357889 15353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:27.357935 15354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:27.358054 15347 server_base.cc:1034] running on GCE node
W20260806 16:03:27.357889 15356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:27.358820 15347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:27.359870 15347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:27.361048 15347 hybrid_clock.cc:648] HybridClock initialized: now 1786032207361022 us; error 39 us; skew 500 ppm
Aug 06 16:03:27 dist-test-slave-tm1g krb5kdc[15334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:27.368991 15347 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:03:27.369982 15347 webserver.cc:460] Webserver started at http://127.0.0.1:43733/ using document root <none> and password file <none>
I20260806 16:03:27.370633 15347 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:27.370730 15347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:27.370968 15347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:27.373075 15347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "2140280d334847a39c2b83c81e158564"
format_stamp: "Formatted at 2026-08-06 16:03:27 on dist-test-slave-tm1g"
server_key: "c28e3d2353d194d2f842dd73a7f2a6cf"
server_key_iv: "3995b9f0afe65d92fd900f4285e84636"
server_key_version: "encryptionkey@0"
I20260806 16:03:27.373663 15347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "2140280d334847a39c2b83c81e158564"
format_stamp: "Formatted at 2026-08-06 16:03:27 on dist-test-slave-tm1g"
server_key: "c28e3d2353d194d2f842dd73a7f2a6cf"
server_key_iv: "3995b9f0afe65d92fd900f4285e84636"
server_key_version: "encryptionkey@0"
I20260806 16:03:27.377502 15347 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 16:03:27.379973 15363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:27.381245 15347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:03:27.381407 15347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "2140280d334847a39c2b83c81e158564"
format_stamp: "Formatted at 2026-08-06 16:03:27 on dist-test-slave-tm1g"
server_key: "c28e3d2353d194d2f842dd73a7f2a6cf"
server_key_iv: "3995b9f0afe65d92fd900f4285e84636"
server_key_version: "encryptionkey@0"
I20260806 16:03:27.381522 15347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:27.403728 15347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:27.406553 15347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:27.406754 15347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:27.413693 15347 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35767
I20260806 16:03:27.413723 15417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35767 every 8 connection(s)
I20260806 16:03:27.414865 15347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:27.417477 15418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:27.422840 15418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564: Bootstrap starting.
I20260806 16:03:27.424968 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 15347
I20260806 16:03:27.425107 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:27.425292 15418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:27.425427 25726 external_mini_cluster.cc:1428] Setting key e8a4170979fbbef8d268f7598dd88ce5
I20260806 16:03:27.426222 15418 log.cc:824] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:27.428118 15418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564: No bootstrap required, opened a new log
I20260806 16:03:27.431468 15418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2140280d334847a39c2b83c81e158564" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35767 } }
I20260806 16:03:27.431668 15418 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:27.431744 15418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2140280d334847a39c2b83c81e158564, State: Initialized, Role: FOLLOWER
I20260806 16:03:27.432191 15418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2140280d334847a39c2b83c81e158564" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35767 } }
I20260806 16:03:27.432310 15418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:27.432394 15418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:27.432507 15418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:27.433429 15418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2140280d334847a39c2b83c81e158564" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35767 } }
I20260806 16:03:27.433763 15418 leader_election.cc:302] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2140280d334847a39c2b83c81e158564; no voters:
I20260806 16:03:27.434080 15418 leader_election.cc:288] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:27.434149 15423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:27.434113 15422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:27.426561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:40981 (local address 127.0.0.1:35767)
0806 16:03:27.427015 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:27.427031 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:27.427065 (+ 34us) server_negotiation.cc:407] Connection header received
0806 16:03:27.427682 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:27.427698 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:27.428010 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:27.428344 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:27.428758 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:27.429691 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:27.430698 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:27.431159 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:27.431337 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:27.432251 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:27.432261 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:27.432273 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:27.432779 (+ 506us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:27.432833 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:27.432856 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:27.433001 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:27.433154 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 16:03:27.433281 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":162,"threads_started":1}
I20260806 16:03:27.434353 15423 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [term 1 LEADER]: Becoming Leader. State: Replica: 2140280d334847a39c2b83c81e158564, State: Running, Role: LEADER
I20260806 16:03:27.434652 15423 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2140280d334847a39c2b83c81e158564" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35767 } }
I20260806 16:03:27.435316 15418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:27.435791 15423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2140280d334847a39c2b83c81e158564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2140280d334847a39c2b83c81e158564" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35767 } } }
I20260806 16:03:27.435967 15423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:27.436236 15423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2140280d334847a39c2b83c81e158564. Latest consensus state: current_term: 1 leader_uuid: "2140280d334847a39c2b83c81e158564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2140280d334847a39c2b83c81e158564" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35767 } } }
I20260806 16:03:27.436390 15423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:27.436780 15429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:27.437269 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:27.439980 15429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:27.445554 15429 catalog_manager.cc:1347] Generated new cluster ID: 57891189592747698270e2911467800d
I20260806 16:03:27.445613 15429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:27.447233 15422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:27.442234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:57871 (local address 127.0.0.1:35767)
0806 16:03:27.442377 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:27.442381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:27.442611 (+ 230us) server_negotiation.cc:407] Connection header received
0806 16:03:27.442724 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:27.442727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:27.442771 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:27.442846 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:27.443159 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:27.443855 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:27.445009 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:27.445402 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:27.445526 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:27.446769 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:27.446772 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:27.446775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:27.446787 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:27.446814 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:27.446826 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:27.446902 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:27.447081 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 16:03:27.447111 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 16:03:27.459909 15429 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:27.460752 15429 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:27.472317 15429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2140280d334847a39c2b83c81e158564: Generated new TSK 0
I20260806 16:03:27.472945 15429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:27.477219 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 15347
W20260806 16:03:27.483731 15332 connection.cc:535] client connection to 127.0.0.1:35767 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35767): Connection reset by peer (error 104)
2026-08-06T16:03:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:27 dist-test-slave-tm1g krb5kdc[15449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:27 dist-test-slave-tm1g krb5kdc[15449](info): set up 2 sockets
Aug 06 16:03:27 dist-test-slave-tm1g krb5kdc[15449](info): commencing operation
krb5kdc: starting...
W20260806 16:03:28.530879 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:28 dist-test-slave-tm1g krb5kdc[15449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 16:03:28.648463 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33711
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:28.754518 15462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:28.754796 15462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:28.754868 15462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:28.758821 15462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:28.758903 15462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:28.758953 15462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:28.759012 15462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:28.759052 15462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:28.764129 15462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33711
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:28.765137 15462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:28.766135 15462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:28.774741 15469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:28.774717 15468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:28.774725 15471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:28.774966 15462 server_base.cc:1034] running on GCE node
I20260806 16:03:28.775596 15462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:28.776648 15462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:28.777814 15462 hybrid_clock.cc:648] HybridClock initialized: now 1786032208777793 us; error 33 us; skew 500 ppm
Aug 06 16:03:28 dist-test-slave-tm1g krb5kdc[15449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032208, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:28.780447 15462 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 16:03:28.781301 15462 webserver.cc:460] Webserver started at http://127.0.0.1:42329/ using document root <none> and password file <none>
I20260806 16:03:28.781831 15462 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:28.781890 15462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:28.782145 15462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:28.784171 15462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "9950ea2478b94435979923491e267f5e"
format_stamp: "Formatted at 2026-08-06 16:03:28 on dist-test-slave-tm1g"
server_key: "51fe0322be93ba0359889a2831fdfb21"
server_key_iv: "bb208001f704ef8b3a773efc8f54ddfc"
server_key_version: "encryptionkey@0"
I20260806 16:03:28.784749 15462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "9950ea2478b94435979923491e267f5e"
format_stamp: "Formatted at 2026-08-06 16:03:28 on dist-test-slave-tm1g"
server_key: "51fe0322be93ba0359889a2831fdfb21"
server_key_iv: "bb208001f704ef8b3a773efc8f54ddfc"
server_key_version: "encryptionkey@0"
I20260806 16:03:28.788321 15462 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 16:03:28.790732 15478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:28.792248 15462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:03:28.792402 15462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "9950ea2478b94435979923491e267f5e"
format_stamp: "Formatted at 2026-08-06 16:03:28 on dist-test-slave-tm1g"
server_key: "51fe0322be93ba0359889a2831fdfb21"
server_key_iv: "bb208001f704ef8b3a773efc8f54ddfc"
server_key_version: "encryptionkey@0"
I20260806 16:03:28.792534 15462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:28.796975 15462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:28.799829 15462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:28.800027 15462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:28.807189 15462 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33711
I20260806 16:03:28.807214 15530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33711 every 8 connection(s)
I20260806 16:03:28.808065 15462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:28.810824 15531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:28.813486 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 15462
I20260806 16:03:28.813657 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:28.813941 25726 external_mini_cluster.cc:1428] Setting key 7bd4290894b9902973a2b0021bd7d10b
I20260806 16:03:28.816473 15531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e: Bootstrap starting.
I20260806 16:03:28.818817 15531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:28.819272 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:28.815337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:42941 (local address 127.0.0.1:33711)
0806 16:03:28.815773 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:28.815787 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:28.815822 (+ 35us) server_negotiation.cc:407] Connection header received
0806 16:03:28.816457 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:28.816473 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:28.816851 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:28.817176 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:28.817965 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:28.817970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:28.817981 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:28.817992 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:28.818045 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:28.818064 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:28.818210 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:28.818278 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 16:03:28.818582 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":138,"threads_started":1}
I20260806 16:03:28.819623 15531 log.cc:824] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:28.821597 15531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e: No bootstrap required, opened a new log
I20260806 16:03:28.821852 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:28.825615 15534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:28.823915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.78:52241 (local address 127.0.0.1:33711)
0806 16:03:28.824020 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:28.824023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:28.824265 (+ 242us) server_negotiation.cc:407] Connection header received
0806 16:03:28.824404 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:28.824406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:28.824435 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:28.824492 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:28.825202 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:28.825205 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:28.825207 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:28.825216 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:28.825240 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:28.825249 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:28.825331 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:28.825439 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 16:03:28.825471 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 16:03:28.825942 15531 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9950ea2478b94435979923491e267f5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33711 } }
I20260806 16:03:28.826176 15531 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:28.826234 15531 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9950ea2478b94435979923491e267f5e, State: Initialized, Role: FOLLOWER
I20260806 16:03:28.826685 15531 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9950ea2478b94435979923491e267f5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33711 } }
I20260806 16:03:28.826843 15531 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:28.826931 15531 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:28.827029 15531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:28.827883 15531 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9950ea2478b94435979923491e267f5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33711 } }
I20260806 16:03:28.828203 15531 leader_election.cc:302] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9950ea2478b94435979923491e267f5e; no voters:
I20260806 16:03:28.828495 15531 leader_election.cc:288] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:28.828570 15541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:28.828835 15541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [term 1 LEADER]: Becoming Leader. State: Replica: 9950ea2478b94435979923491e267f5e, State: Running, Role: LEADER
I20260806 16:03:28.829157 15541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9950ea2478b94435979923491e267f5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33711 } }
I20260806 16:03:28.829618 15531 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:28.830828 15544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9950ea2478b94435979923491e267f5e. Latest consensus state: current_term: 1 leader_uuid: "9950ea2478b94435979923491e267f5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9950ea2478b94435979923491e267f5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33711 } } }
I20260806 16:03:28.830965 15544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:28.831404 15542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9950ea2478b94435979923491e267f5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9950ea2478b94435979923491e267f5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33711 } } }
I20260806 16:03:28.831513 15542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e [sys.catalog]: This master's current role is: LEADER
W20260806 16:03:28.834242 15557 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:03:28.834350 15557 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:03:28.834436 15551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:28.835099 15551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:28.839974 15551 catalog_manager.cc:1347] Generated new cluster ID: 3f92334584934d6782aee6666ceaa296
I20260806 16:03:28.840046 15551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:28.870313 15551 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:28.870675 15551 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:28.881340 15551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9950ea2478b94435979923491e267f5e: Generated new TSK 0
I20260806 16:03:28.881979 15551 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:28.894883 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 15462
2026-08-06T16:03:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1417 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:28 dist-test-slave-tm1g krb5kdc[15563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:28 dist-test-slave-tm1g krb5kdc[15563](info): set up 2 sockets
Aug 06 16:03:28 dist-test-slave-tm1g krb5kdc[15563](info): commencing operation
krb5kdc: starting...
W20260806 16:03:29.954695 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:30 dist-test-slave-tm1g krb5kdc[15563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:03:30.073127 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:43675
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:41823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:43675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:30.181396 15576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:30.181632 15576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:30.181672 15576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:30.185621 15576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:30.185678 15576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:30.185696 15576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:30.185712 15576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:30.185727 15576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:30.190979 15576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:41823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:43675
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:43675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:30.191941 15576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:30.192909 15576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:30.201043 15581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:30.201056 15584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:30.201098 15582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:30.201234 15576 server_base.cc:1034] running on GCE node
I20260806 16:03:30.201907 15576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:30.202996 15576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:30.204188 15576 hybrid_clock.cc:648] HybridClock initialized: now 1786032210204161 us; error 40 us; skew 500 ppm
Aug 06 16:03:30 dist-test-slave-tm1g krb5kdc[15563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032210, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:30.206876 15576 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:03:30.207646 15576 webserver.cc:460] Webserver started at http://127.25.31.190:43355/ using document root <none> and password file <none>
I20260806 16:03:30.208139 15576 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:30.208195 15576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:30.208352 15576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:30.210253 15576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "940309c65cb344859f6d48e40d7848e6"
format_stamp: "Formatted at 2026-08-06 16:03:30 on dist-test-slave-tm1g"
server_key: "4968beba215c196a343e7dffb0be7a1d"
server_key_iv: "a7fd10240e05735674a8372beed2c822"
server_key_version: "encryptionkey@0"
I20260806 16:03:30.210817 15576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "940309c65cb344859f6d48e40d7848e6"
format_stamp: "Formatted at 2026-08-06 16:03:30 on dist-test-slave-tm1g"
server_key: "4968beba215c196a343e7dffb0be7a1d"
server_key_iv: "a7fd10240e05735674a8372beed2c822"
server_key_version: "encryptionkey@0"
I20260806 16:03:30.214437 15576 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:03:30.216646 15591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:30.217815 15576 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:03:30.217923 15576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "940309c65cb344859f6d48e40d7848e6"
format_stamp: "Formatted at 2026-08-06 16:03:30 on dist-test-slave-tm1g"
server_key: "4968beba215c196a343e7dffb0be7a1d"
server_key_iv: "a7fd10240e05735674a8372beed2c822"
server_key_version: "encryptionkey@0"
I20260806 16:03:30.218005 15576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:30.234403 15576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:30.236937 15576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:30.237078 15576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:30.243847 15576 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:43675
I20260806 16:03:30.243875 15643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:43675 every 8 connection(s)
I20260806 16:03:30.244724 15576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:30.247320 15644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:30.248903 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 15576
I20260806 16:03:30.249044 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:30.249329 25726 external_mini_cluster.cc:1428] Setting key 634294900b7633401e1457d59a945037
I20260806 16:03:30.253845 15644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6: Bootstrap starting.
I20260806 16:03:30.256330 15644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:30.257225 15644 log.cc:824] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6: Log is configured to *not* fsync() on all Append() calls
Aug 06 16:03:30 dist-test-slave-tm1g krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:03:30.259282 15644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6: No bootstrap required, opened a new log
I20260806 16:03:30.263451 15644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "940309c65cb344859f6d48e40d7848e6" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43675 } }
I20260806 16:03:30.263725 15644 raft_consensus.cc:374] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:30.263775 15644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 940309c65cb344859f6d48e40d7848e6, State: Initialized, Role: FOLLOWER
I20260806 16:03:30.264235 15644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "940309c65cb344859f6d48e40d7848e6" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43675 } }
I20260806 16:03:30.264416 15644 raft_consensus.cc:388] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:30.264523 15644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:30.264642 15644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:30.265492 15644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "940309c65cb344859f6d48e40d7848e6" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43675 } }
I20260806 16:03:30.265682 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:30.250665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57787 (local address 127.25.31.190:43675)
0806 16:03:30.251076 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:30.251096 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:30.251137 (+ 41us) server_negotiation.cc:407] Connection header received
0806 16:03:30.252006 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:30.252029 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:30.252473 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:30.252822 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:30.253472 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:30.254948 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:30.255804 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:30.256495 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:30.256662 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:30.259706 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:30.259712 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:30.259727 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:30.259756 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:30.262335 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:30.262841 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:30.262847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:30.262855 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:30.262996 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:30.263358 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:30.263361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:30.263363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:30.263884 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:30.264103 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:30.264467 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 16:03:30.264855 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":99,"threads_started":1}
I20260806 16:03:30.265854 15644 leader_election.cc:302] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 940309c65cb344859f6d48e40d7848e6; no voters:
I20260806 16:03:30.266206 15644 leader_election.cc:288] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:30.266330 15649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:30.266544 15649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [term 1 LEADER]: Becoming Leader. State: Replica: 940309c65cb344859f6d48e40d7848e6, State: Running, Role: LEADER
I20260806 16:03:30.266927 15649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "940309c65cb344859f6d48e40d7848e6" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43675 } }
I20260806 16:03:30.267406 15644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:30.268652 15651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 940309c65cb344859f6d48e40d7848e6. Latest consensus state: current_term: 1 leader_uuid: "940309c65cb344859f6d48e40d7848e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "940309c65cb344859f6d48e40d7848e6" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43675 } } }
I20260806 16:03:30.268937 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:30.269078 15651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:30.268961 15650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "940309c65cb344859f6d48e40d7848e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "940309c65cb344859f6d48e40d7848e6" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43675 } } }
I20260806 16:03:30.269315 15650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6 [sys.catalog]: This master's current role is: LEADER
W20260806 16:03:30.273176 15664 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:03:30.273257 15664 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:03:30.273342 15662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:30.273995 15662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:30.280678 15662 catalog_manager.cc:1347] Generated new cluster ID: a461eacb56704e9e85dc59acc56cf316
I20260806 16:03:30.280735 15662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:30.282877 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:30.274727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52187 (local address 127.25.31.190:43675)
0806 16:03:30.274843 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:30.274847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:30.275223 (+ 376us) server_negotiation.cc:407] Connection header received
0806 16:03:30.275329 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:30.275333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:30.275384 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:30.275481 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:30.276200 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:30.276711 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:30.277527 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:30.277757 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:30.277866 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:30.279318 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:30.279321 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:30.279324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:30.279355 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:30.280954 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:30.281546 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:30.281551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:30.281553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:30.281629 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:30.281956 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:30.281961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:30.281962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:30.282141 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:30.282250 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:30.282651 (+ 401us) server_negotiation.cc:299] Negotiation successful
0806 16:03:30.282738 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 16:03:30.313064 15662 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:30.314085 15662 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:30.324743 15662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 940309c65cb344859f6d48e40d7848e6: Generated new TSK 0
I20260806 16:03:30.325544 15662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:30.337886 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 15576
W20260806 16:03:30.344344 15561 connection.cc:535] client connection to 127.25.31.190:43675 recv error: Network error: failed to read from TLS socket (remote: 127.25.31.190:43675): Connection reset by peer (error 104)
2026-08-06T16:03:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:30 dist-test-slave-tm1g krb5kdc[15675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:30 dist-test-slave-tm1g krb5kdc[15675](info): set up 2 sockets
Aug 06 16:03:30 dist-test-slave-tm1g krb5kdc[15675](info): commencing operation
krb5kdc: starting...
W20260806 16:03:31.387308 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:31 dist-test-slave-tm1g krb5kdc[15675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:03:31.505625 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:43279
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:43279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:31.615315 15689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:31.615561 15689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:31.615602 15689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:31.619812 15689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:31.619867 15689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:31.619887 15689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:31.619904 15689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:31.619956 15689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:31.625061 15689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:43279
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:43279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:31.626021 15689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:31.627046 15689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:31.635689 15694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:31.635711 15689 server_base.cc:1034] running on GCE node
W20260806 16:03:31.635704 15697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:31.635881 15695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:31.636420 15689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:31.637387 15689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:31.638541 15689 hybrid_clock.cc:648] HybridClock initialized: now 1786032211638513 us; error 43 us; skew 500 ppm
Aug 06 16:03:31 dist-test-slave-tm1g krb5kdc[15675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032211, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:31.641160 15689 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:03:31.642022 15689 webserver.cc:460] Webserver started at http://127.25.31.190:33779/ using document root <none> and password file <none>
I20260806 16:03:31.642620 15689 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:31.642709 15689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:31.642932 15689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:31.644876 15689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "922be6d15d874273b7349b0b759a87ba"
format_stamp: "Formatted at 2026-08-06 16:03:31 on dist-test-slave-tm1g"
server_key: "29f95f90ceb30be05bb95fad4dada06b"
server_key_iv: "2b2c90c952dfe0109f93746c12789f85"
server_key_version: "encryptionkey@0"
I20260806 16:03:31.645444 15689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "922be6d15d874273b7349b0b759a87ba"
format_stamp: "Formatted at 2026-08-06 16:03:31 on dist-test-slave-tm1g"
server_key: "29f95f90ceb30be05bb95fad4dada06b"
server_key_iv: "2b2c90c952dfe0109f93746c12789f85"
server_key_version: "encryptionkey@0"
I20260806 16:03:31.649024 15689 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 16:03:31.651360 15704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:31.652451 15689 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:03:31.652593 15689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "922be6d15d874273b7349b0b759a87ba"
format_stamp: "Formatted at 2026-08-06 16:03:31 on dist-test-slave-tm1g"
server_key: "29f95f90ceb30be05bb95fad4dada06b"
server_key_iv: "2b2c90c952dfe0109f93746c12789f85"
server_key_version: "encryptionkey@0"
I20260806 16:03:31.652701 15689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:31.682801 15689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:31.685662 15689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:31.685871 15689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:31.692857 15689 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:43279
I20260806 16:03:31.692886 15756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:43279 every 8 connection(s)
I20260806 16:03:31.693830 15689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:31.696516 15757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:31.701429 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 15689
I20260806 16:03:31.701594 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:31.701859 25726 external_mini_cluster.cc:1428] Setting key 03d375bae49921ca7193758767878a41
I20260806 16:03:31.701891 15757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba: Bootstrap starting.
I20260806 16:03:31.704298 15757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:31.705062 15757 log.cc:824] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:31.707178 15757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba: No bootstrap required, opened a new log
I20260806 16:03:31.711020 15760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:31.703278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57959 (local address 127.25.31.190:43279)
0806 16:03:31.703681 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:31.703704 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:31.703739 (+ 35us) server_negotiation.cc:407] Connection header received
0806 16:03:31.704400 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:31.704424 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:31.704764 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:31.705037 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:31.705431 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:31.706649 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:31.707827 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:31.708273 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:31.708400 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:31.709356 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:31.709362 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:31.709374 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:31.709739 (+ 365us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:31.709790 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:31.709812 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:31.709953 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:31.710139 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 16:03:31.710268 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":129,"threads_started":1}
I20260806 16:03:31.711113 15757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922be6d15d874273b7349b0b759a87ba" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43279 } }
I20260806 16:03:31.711377 15757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:31.711454 15757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922be6d15d874273b7349b0b759a87ba, State: Initialized, Role: FOLLOWER
I20260806 16:03:31.711853 15757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922be6d15d874273b7349b0b759a87ba" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43279 } }
I20260806 16:03:31.711997 15757 raft_consensus.cc:388] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:31.712078 15757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:31.712172 15757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:31.713209 15757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922be6d15d874273b7349b0b759a87ba" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43279 } }
I20260806 16:03:31.713573 15757 leader_election.cc:302] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 922be6d15d874273b7349b0b759a87ba; no voters:
I20260806 16:03:31.713889 15757 leader_election.cc:288] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:31.713956 15762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:31.714051 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:31.714152 15762 raft_consensus.cc:686] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [term 1 LEADER]: Becoming Leader. State: Replica: 922be6d15d874273b7349b0b759a87ba, State: Running, Role: LEADER
I20260806 16:03:31.714454 15762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922be6d15d874273b7349b0b759a87ba" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43279 } }
I20260806 16:03:31.715124 15757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:31.715593 15762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "922be6d15d874273b7349b0b759a87ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922be6d15d874273b7349b0b759a87ba" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43279 } } }
I20260806 16:03:31.715703 15762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:31.715996 15764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 922be6d15d874273b7349b0b759a87ba. Latest consensus state: current_term: 1 leader_uuid: "922be6d15d874273b7349b0b759a87ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922be6d15d874273b7349b0b759a87ba" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43279 } } }
I20260806 16:03:31.716091 15764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba [sys.catalog]: This master's current role is: LEADER
W20260806 16:03:31.720468 15782 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:03:31.720547 15782 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:03:31.720624 15771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:31.721536 15760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:31.716678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39045 (local address 127.25.31.190:43279)
0806 16:03:31.716767 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:31.716771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:31.717001 (+ 230us) server_negotiation.cc:407] Connection header received
0806 16:03:31.717109 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:31.717113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:31.717164 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:31.717241 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:31.717821 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:31.718593 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:31.719703 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:31.720032 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:31.720138 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:31.721033 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:31.721037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:31.721040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:31.721054 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:31.721080 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:31.721094 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:31.721187 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:31.721380 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 16:03:31.721414 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 16:03:31.721575 15771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:31.726246 15771 catalog_manager.cc:1347] Generated new cluster ID: f0618022ffc54a0e8e754637930f01eb
I20260806 16:03:31.726331 15771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:31.738566 15771 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:31.739413 15771 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:31.745779 15771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 922be6d15d874273b7349b0b759a87ba: Generated new TSK 0
I20260806 16:03:31.746623 15771 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:31.757436 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 15689
W20260806 16:03:31.763643 15673 connection.cc:535] client connection to 127.25.31.190:43279 recv error: Network error: failed to read from TLS socket (remote: 127.25.31.190:43279): Connection reset by peer (error 104)
2026-08-06T16:03:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1419 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:31 dist-test-slave-tm1g krb5kdc[15788](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:31 dist-test-slave-tm1g krb5kdc[15788](info): set up 2 sockets
Aug 06 16:03:31 dist-test-slave-tm1g krb5kdc[15788](info): commencing operation
krb5kdc: starting...
W20260806 16:03:32.807049 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:32 dist-test-slave-tm1g krb5kdc[15788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:03:32.925212 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:41295
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:33771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:41295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:33.034749 15801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:33.034989 15801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:33.035029 15801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:33.038899 15801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:33.038950 15801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:33.038969 15801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:33.038986 15801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:33.039000 15801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:33.044039 15801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:33771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:41295
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:41295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:33.044946 15801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:33.045883 15801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:33.054033 15809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:33.054042 15807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:33.054042 15806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:33.054394 15801 server_base.cc:1034] running on GCE node
I20260806 16:03:33.054982 15801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:33.056010 15801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:33.057222 15801 hybrid_clock.cc:648] HybridClock initialized: now 1786032213057198 us; error 41 us; skew 500 ppm
Aug 06 16:03:33 dist-test-slave-tm1g krb5kdc[15788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032213, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:33.063702 15801 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:03:33.064568 15801 webserver.cc:460] Webserver started at http://127.25.31.190:42091/ using document root <none> and password file <none>
I20260806 16:03:33.065129 15801 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:33.065222 15801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:33.065474 15801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:33.067559 15801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "408a3fd37f3e447b9791ce549bf19f00"
format_stamp: "Formatted at 2026-08-06 16:03:33 on dist-test-slave-tm1g"
server_key: "2706fabe4bd9448ef4d57e69965a2cd2"
server_key_iv: "f0e183aecbc71d4645d87f5a83ea92a3"
server_key_version: "encryptionkey@0"
I20260806 16:03:33.068127 15801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "408a3fd37f3e447b9791ce549bf19f00"
format_stamp: "Formatted at 2026-08-06 16:03:33 on dist-test-slave-tm1g"
server_key: "2706fabe4bd9448ef4d57e69965a2cd2"
server_key_iv: "f0e183aecbc71d4645d87f5a83ea92a3"
server_key_version: "encryptionkey@0"
I20260806 16:03:33.071735 15801 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:03:33.074039 15816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:33.075225 15801 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:03:33.075366 15801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "408a3fd37f3e447b9791ce549bf19f00"
format_stamp: "Formatted at 2026-08-06 16:03:33 on dist-test-slave-tm1g"
server_key: "2706fabe4bd9448ef4d57e69965a2cd2"
server_key_iv: "f0e183aecbc71d4645d87f5a83ea92a3"
server_key_version: "encryptionkey@0"
I20260806 16:03:33.075474 15801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:33.079985 15801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:33.083134 15801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:33.083330 15801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:33.090236 15801 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:41295
I20260806 16:03:33.090263 15869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:41295 every 8 connection(s)
I20260806 16:03:33.091161 15801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:33.093770 15870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:33.099105 15870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00: Bootstrap starting.
I20260806 16:03:33.101055 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 15801
I20260806 16:03:33.101194 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:33.101501 25726 external_mini_cluster.cc:1428] Setting key 0d2cd09461f36ea4deff5443bc7006f8
I20260806 16:03:33.101569 15870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:33.102512 15870 log.cc:824] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:33.104400 15870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00: No bootstrap required, opened a new log
I20260806 16:03:33.106410 15874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:33.102652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51309 (local address 127.25.31.190:41295)
0806 16:03:33.103157 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:33.103172 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:33.103206 (+ 34us) server_negotiation.cc:407] Connection header received
0806 16:03:33.103819 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:33.103835 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:33.104091 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:33.104355 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:33.105165 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:33.105169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:33.105180 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:33.105189 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:33.105236 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:33.105254 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:33.105349 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:33.105497 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 16:03:33.105666 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"spinlock_wait_cycles":21760,"thread_start_us":202,"threads_started":1}
I20260806 16:03:33.108613 15870 raft_consensus.cc:348] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408a3fd37f3e447b9791ce549bf19f00" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41295 } }
I20260806 16:03:33.108896 15870 raft_consensus.cc:374] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:33.108958 15870 raft_consensus.cc:729] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408a3fd37f3e447b9791ce549bf19f00, State: Initialized, Role: FOLLOWER
I20260806 16:03:33.109216 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:33.109413 15870 consensus_queue.cc:260] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408a3fd37f3e447b9791ce549bf19f00" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41295 } }
I20260806 16:03:33.109536 15870 raft_consensus.cc:388] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:33.109617 15870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:33.109711 15870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:33.110633 15870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408a3fd37f3e447b9791ce549bf19f00" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41295 } }
I20260806 16:03:33.110968 15870 leader_election.cc:302] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 408a3fd37f3e447b9791ce549bf19f00; no voters:
I20260806 16:03:33.111299 15870 leader_election.cc:288] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:33.111356 15879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:33.111646 15879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [term 1 LEADER]: Becoming Leader. State: Replica: 408a3fd37f3e447b9791ce549bf19f00, State: Running, Role: LEADER
I20260806 16:03:33.111994 15879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408a3fd37f3e447b9791ce549bf19f00" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41295 } }
I20260806 16:03:33.112706 15870 sys_catalog.cc:564] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:33.113651 15882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "408a3fd37f3e447b9791ce549bf19f00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408a3fd37f3e447b9791ce549bf19f00" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41295 } } }
I20260806 16:03:33.113771 15882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:33.114109 15881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 408a3fd37f3e447b9791ce549bf19f00. Latest consensus state: current_term: 1 leader_uuid: "408a3fd37f3e447b9791ce549bf19f00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408a3fd37f3e447b9791ce549bf19f00" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 41295 } } }
I20260806 16:03:33.114234 15881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:33.114152 15889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:33.116501 15874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:33.111387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34803 (local address 127.25.31.190:41295)
0806 16:03:33.111638 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:33.111642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:33.115012 (+ 3370us) server_negotiation.cc:407] Connection header received
0806 16:03:33.115204 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:33.115207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:33.115244 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:33.115343 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:33.116096 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:33.116100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:33.116102 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:33.116114 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:33.116139 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:33.116151 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:33.116239 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:33.116350 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 16:03:33.116386 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260806 16:03:33.117672 15889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:33.122830 15889 catalog_manager.cc:1347] Generated new cluster ID: 9982ade02cce4601a461491f6a19ae5d
I20260806 16:03:33.122893 15889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:33.133257 15889 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:33.133538 15889 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:33.142778 15889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 408a3fd37f3e447b9791ce549bf19f00: Generated new TSK 0
I20260806 16:03:33.143406 15889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:33.151576 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 15801
2026-08-06T16:03:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1393 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:33 dist-test-slave-tm1g krb5kdc[15901](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:33 dist-test-slave-tm1g krb5kdc[15901](info): set up 2 sockets
Aug 06 16:03:33 dist-test-slave-tm1g krb5kdc[15901](info): commencing operation
krb5kdc: starting...
W20260806 16:03:34.194989 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:34 dist-test-slave-tm1g krb5kdc[15901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:03:34.313337 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:43619
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:36665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:43619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:34.423559 15914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:34.423869 15914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:34.423943 15914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:34.428166 15914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:34.428267 15914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:34.428308 15914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:34.428378 15914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:34.428438 15914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:34.433843 15914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:36665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:43619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:43619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:34.435070 15914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:34.436125 15914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:34.444757 15922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:34.444777 15919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:34.444756 15920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:34.444855 15914 server_base.cc:1034] running on GCE node
I20260806 16:03:34.445591 15914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:34.446731 15914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:34.447935 15914 hybrid_clock.cc:648] HybridClock initialized: now 1786032214447909 us; error 45 us; skew 500 ppm
Aug 06 16:03:34 dist-test-slave-tm1g krb5kdc[15901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032214, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:34.450914 15914 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:03:34.451781 15914 webserver.cc:460] Webserver started at http://127.25.31.190:39923/ using document root <none> and password file <none>
I20260806 16:03:34.452332 15914 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:34.452445 15914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:34.452677 15914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:34.454738 15914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "a8bf5c0973fc4478b41a1c4e29b59581"
format_stamp: "Formatted at 2026-08-06 16:03:34 on dist-test-slave-tm1g"
server_key: "1b29e59fc97604c28f3610a0f3da5ec0"
server_key_iv: "4fb7f5d8001fba5fe526d6a4d9533768"
server_key_version: "encryptionkey@0"
I20260806 16:03:34.455300 15914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "a8bf5c0973fc4478b41a1c4e29b59581"
format_stamp: "Formatted at 2026-08-06 16:03:34 on dist-test-slave-tm1g"
server_key: "1b29e59fc97604c28f3610a0f3da5ec0"
server_key_iv: "4fb7f5d8001fba5fe526d6a4d9533768"
server_key_version: "encryptionkey@0"
I20260806 16:03:34.458882 15914 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:03:34.461308 15929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:34.462628 15914 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:03:34.462772 15914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "a8bf5c0973fc4478b41a1c4e29b59581"
format_stamp: "Formatted at 2026-08-06 16:03:34 on dist-test-slave-tm1g"
server_key: "1b29e59fc97604c28f3610a0f3da5ec0"
server_key_iv: "4fb7f5d8001fba5fe526d6a4d9533768"
server_key_version: "encryptionkey@0"
I20260806 16:03:34.462880 15914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:34.475831 15914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:34.478701 15914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:34.478900 15914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:34.485962 15914 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:43619
I20260806 16:03:34.485996 15982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:43619 every 8 connection(s)
I20260806 16:03:34.486899 15914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:34.489650 15983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:34.489699 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 15914
I20260806 16:03:34.489842 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:34.490145 25726 external_mini_cluster.cc:1428] Setting key 3103cfb5e35c2ee8a51c3a8ad9f074ea
I20260806 16:03:34.497011 15983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581: Bootstrap starting.
I20260806 16:03:34.499691 15983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581: Neither blocks nor log segments found. Creating new log.
Aug 06 16:03:34 dist-test-slave-tm1g krb5kdc[15901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:03:34.500593 15983 log.cc:824] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:34.502744 15983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581: No bootstrap required, opened a new log
I20260806 16:03:34.505563 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:34.491558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43593 (local address 127.25.31.190:43619)
0806 16:03:34.492030 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:34.492048 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:34.492093 (+ 45us) server_negotiation.cc:407] Connection header received
0806 16:03:34.493188 (+ 1095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:34.493213 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:34.493649 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:34.494031 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:34.494816 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:34.496324 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:34.497310 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:34.497980 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:34.498155 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:34.500971 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:34.500977 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:34.500995 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:34.501030 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:34.502841 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:34.503290 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:34.503296 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:34.503308 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:34.503420 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:34.503701 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:34.503705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:34.503706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:34.504095 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:34.504295 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:34.504590 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 16:03:34.504867 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":125,"threads_started":1}
I20260806 16:03:34.506412 15983 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8bf5c0973fc4478b41a1c4e29b59581" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43619 } }
I20260806 16:03:34.506693 15983 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:34.506753 15983 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8bf5c0973fc4478b41a1c4e29b59581, State: Initialized, Role: FOLLOWER
I20260806 16:03:34.507200 15983 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8bf5c0973fc4478b41a1c4e29b59581" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43619 } }
I20260806 16:03:34.507380 15983 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:34.507457 15983 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:34.507591 15983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:34.508528 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:34.508747 15983 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8bf5c0973fc4478b41a1c4e29b59581" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43619 } }
I20260806 16:03:34.509122 15983 leader_election.cc:302] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8bf5c0973fc4478b41a1c4e29b59581; no voters:
I20260806 16:03:34.509469 15983 leader_election.cc:288] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:34.509586 15989 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:34.509843 15989 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [term 1 LEADER]: Becoming Leader. State: Replica: a8bf5c0973fc4478b41a1c4e29b59581, State: Running, Role: LEADER
I20260806 16:03:34.510221 15989 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8bf5c0973fc4478b41a1c4e29b59581" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43619 } }
I20260806 16:03:34.510793 15983 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:34.511446 15990 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8bf5c0973fc4478b41a1c4e29b59581. Latest consensus state: current_term: 1 leader_uuid: "a8bf5c0973fc4478b41a1c4e29b59581" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8bf5c0973fc4478b41a1c4e29b59581" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43619 } } }
I20260806 16:03:34.511586 15990 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:34.512269 15991 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8bf5c0973fc4478b41a1c4e29b59581" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8bf5c0973fc4478b41a1c4e29b59581" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 43619 } } }
I20260806 16:03:34.512408 15991 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:34.513581 16002 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:34.516551 16002 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:34.520001 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:34.510792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47549 (local address 127.25.31.190:43619)
0806 16:03:34.510904 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:34.510908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:34.511499 (+ 591us) server_negotiation.cc:407] Connection header received
0806 16:03:34.511823 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:34.511829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:34.511874 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:34.511924 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:34.513017 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:34.513821 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:34.514855 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:34.515204 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:34.515321 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:34.516663 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:34.516666 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:34.516669 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:34.516694 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:34.518162 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:34.518685 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:34.518688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:34.518690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:34.518779 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:34.519093 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:34.519096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:34.519098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:34.519316 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:34.519460 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:34.519732 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 16:03:34.519851 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 16:03:34.522732 16002 catalog_manager.cc:1347] Generated new cluster ID: e5c98ed9c24c4c9fa4589dd337a9927d
I20260806 16:03:34.522795 16002 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:34.532256 16002 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:34.533087 16002 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:34.538432 16002 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8bf5c0973fc4478b41a1c4e29b59581: Generated new TSK 0
I20260806 16:03:34.539158 16002 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:34.548061 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 15914
2026-08-06T16:03:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:34 dist-test-slave-tm1g krb5kdc[16016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:34 dist-test-slave-tm1g krb5kdc[16016](info): set up 2 sockets
Aug 06 16:03:34 dist-test-slave-tm1g krb5kdc[16016](info): commencing operation
krb5kdc: starting...
W20260806 16:03:35.592511 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:35 dist-test-slave-tm1g krb5kdc[16016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:03:35.711310 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:35307
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:35307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:35.820526 16029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:35.820827 16029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:35.820904 16029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:35.824843 16029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:35.824939 16029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:35.824980 16029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:35.825038 16029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:35.825076 16029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:35.830331 16029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:35307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:35307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:35.831386 16029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:35.832419 16029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:35.840739 16037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:35.840764 16035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:35.840982 16029 server_base.cc:1034] running on GCE node
W20260806 16:03:35.841100 16034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:35.841617 16029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:35.842736 16029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:35.843904 16029 hybrid_clock.cc:648] HybridClock initialized: now 1786032215843878 us; error 37 us; skew 500 ppm
Aug 06 16:03:35 dist-test-slave-tm1g krb5kdc[16016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032215, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:35.846659 16029 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:03:35.847515 16029 webserver.cc:460] Webserver started at http://127.25.31.190:44921/ using document root <none> and password file <none>
I20260806 16:03:35.848055 16029 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:35.848114 16029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:35.848366 16029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:35.850404 16029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "08da0d5482654caa9167fbe1d23da7a9"
format_stamp: "Formatted at 2026-08-06 16:03:35 on dist-test-slave-tm1g"
server_key: "e4f930ea1d3105a77aee213bba2601fc"
server_key_iv: "6ae0dcfbc2a60844da18ae5caa6e022e"
server_key_version: "encryptionkey@0"
I20260806 16:03:35.850975 16029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "08da0d5482654caa9167fbe1d23da7a9"
format_stamp: "Formatted at 2026-08-06 16:03:35 on dist-test-slave-tm1g"
server_key: "e4f930ea1d3105a77aee213bba2601fc"
server_key_iv: "6ae0dcfbc2a60844da18ae5caa6e022e"
server_key_version: "encryptionkey@0"
I20260806 16:03:35.854482 16029 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 16:03:35.856714 16044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:35.857903 16029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:03:35.858055 16029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "08da0d5482654caa9167fbe1d23da7a9"
format_stamp: "Formatted at 2026-08-06 16:03:35 on dist-test-slave-tm1g"
server_key: "e4f930ea1d3105a77aee213bba2601fc"
server_key_iv: "6ae0dcfbc2a60844da18ae5caa6e022e"
server_key_version: "encryptionkey@0"
I20260806 16:03:35.858160 16029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:35.892776 16029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:35.895709 16029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:35.895917 16029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:35.902531 16029 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:35307
I20260806 16:03:35.902611 16096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:35307 every 8 connection(s)
I20260806 16:03:35.903476 16029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:35.906090 16097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:35.907048 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 16029
I20260806 16:03:35.907176 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:35.907497 25726 external_mini_cluster.cc:1428] Setting key ced31ac0371b2f8d50c40b11900c2bd6
I20260806 16:03:35.912292 16097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9: Bootstrap starting.
I20260806 16:03:35.914821 16097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:35.915618 16097 log.cc:824] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:35.916087 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:35.908758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50203 (local address 127.25.31.190:35307)
0806 16:03:35.909158 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:35.909173 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:35.909207 (+ 34us) server_negotiation.cc:407] Connection header received
0806 16:03:35.909826 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:35.909841 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:35.910166 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:35.910496 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:35.910810 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:35.911766 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:35.912927 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:35.913369 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:35.913500 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:35.914290 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:35.914295 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:35.914356 (+ 61us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:35.914730 (+ 374us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:35.914782 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:35.914804 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:35.914880 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:35.915067 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 16:03:35.915286 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":131,"threads_started":1}
I20260806 16:03:35.917694 16097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9: No bootstrap required, opened a new log
I20260806 16:03:35.918967 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:35.922443 16097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08da0d5482654caa9167fbe1d23da7a9" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35307 } }
I20260806 16:03:35.922678 16097 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:35.922739 16097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08da0d5482654caa9167fbe1d23da7a9, State: Initialized, Role: FOLLOWER
I20260806 16:03:35.923182 16097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08da0d5482654caa9167fbe1d23da7a9" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35307 } }
I20260806 16:03:35.923332 16097 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:35.923399 16097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:35.923487 16097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:35.924602 16097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08da0d5482654caa9167fbe1d23da7a9" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35307 } }
I20260806 16:03:35.924983 16097 leader_election.cc:302] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08da0d5482654caa9167fbe1d23da7a9; no voters:
I20260806 16:03:35.925251 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:35.921186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52645 (local address 127.25.31.190:35307)
0806 16:03:35.921345 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:35.921348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:35.921439 (+ 91us) server_negotiation.cc:407] Connection header received
0806 16:03:35.921561 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:35.921563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:35.921597 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:35.921653 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:35.921956 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:35.922575 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:35.923665 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:35.923905 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:35.923987 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:35.924769 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:35.924773 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:35.924775 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:35.924788 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:35.924826 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:35.924837 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:35.924895 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:35.925030 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 16:03:35.925076 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 16:03:35.925351 16097 leader_election.cc:288] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:35.925411 16107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:35.925603 16107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [term 1 LEADER]: Becoming Leader. State: Replica: 08da0d5482654caa9167fbe1d23da7a9, State: Running, Role: LEADER
I20260806 16:03:35.925953 16107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08da0d5482654caa9167fbe1d23da7a9" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35307 } }
I20260806 16:03:35.926604 16097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:35.927649 16108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08da0d5482654caa9167fbe1d23da7a9. Latest consensus state: current_term: 1 leader_uuid: "08da0d5482654caa9167fbe1d23da7a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08da0d5482654caa9167fbe1d23da7a9" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35307 } } }
I20260806 16:03:35.927790 16108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:35.928136 16109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08da0d5482654caa9167fbe1d23da7a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08da0d5482654caa9167fbe1d23da7a9" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 35307 } } }
I20260806 16:03:35.928202 16116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:35.928270 16109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:35.931288 16116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:35.936429 16116 catalog_manager.cc:1347] Generated new cluster ID: 4a53c5cc9c0648d8a115cba6e86ae7fc
I20260806 16:03:35.936492 16116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:35.941998 16116 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:35.942920 16116 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:35.946825 16116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08da0d5482654caa9167fbe1d23da7a9: Generated new TSK 0
I20260806 16:03:35.947403 16116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:35.952515 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 16029
2026-08-06T16:03:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:36 dist-test-slave-tm1g krb5kdc[16128](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:36 dist-test-slave-tm1g krb5kdc[16128](info): set up 2 sockets
Aug 06 16:03:36 dist-test-slave-tm1g krb5kdc[16128](info): commencing operation
krb5kdc: starting...
W20260806 16:03:37.004555 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:37 dist-test-slave-tm1g krb5kdc[16128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:03:37.121429 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:36955
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:36955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:37.230371 16142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:37.230664 16142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:37.230746 16142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:37.234692 16142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:37.234743 16142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:37.234763 16142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:37.234779 16142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:37.234795 16142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:37.239848 16142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:36955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:36955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:37.240777 16142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:37.241696 16142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:37.250041 16148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:37.250058 16150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:37.250051 16147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:37.250111 16142 server_base.cc:1034] running on GCE node
I20260806 16:03:37.250798 16142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:37.251789 16142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:37.253005 16142 hybrid_clock.cc:648] HybridClock initialized: now 1786032217252995 us; error 28 us; skew 500 ppm
Aug 06 16:03:37 dist-test-slave-tm1g krb5kdc[16128](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032217, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:37.255762 16142 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:03:37.256621 16142 webserver.cc:460] Webserver started at http://127.25.31.190:33607/ using document root <none> and password file <none>
I20260806 16:03:37.257180 16142 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:37.257273 16142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:37.257501 16142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:37.259549 16142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "5054ce08ba12459d9c900d883b166c30"
format_stamp: "Formatted at 2026-08-06 16:03:37 on dist-test-slave-tm1g"
server_key: "b15d099c1b608ee886d4f094ec43f41b"
server_key_iv: "9b7d567f505271a280d4d1452cf2f0d6"
server_key_version: "encryptionkey@0"
I20260806 16:03:37.260116 16142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "5054ce08ba12459d9c900d883b166c30"
format_stamp: "Formatted at 2026-08-06 16:03:37 on dist-test-slave-tm1g"
server_key: "b15d099c1b608ee886d4f094ec43f41b"
server_key_iv: "9b7d567f505271a280d4d1452cf2f0d6"
server_key_version: "encryptionkey@0"
I20260806 16:03:37.263700 16142 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 16:03:37.265991 16157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:37.267211 16142 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 16:03:37.267357 16142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "5054ce08ba12459d9c900d883b166c30"
format_stamp: "Formatted at 2026-08-06 16:03:37 on dist-test-slave-tm1g"
server_key: "b15d099c1b608ee886d4f094ec43f41b"
server_key_iv: "9b7d567f505271a280d4d1452cf2f0d6"
server_key_version: "encryptionkey@0"
I20260806 16:03:37.267467 16142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:37.271761 16142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:37.274578 16142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:37.274762 16142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:37.281340 16142 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:36955
I20260806 16:03:37.281407 16211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:36955 every 8 connection(s)
I20260806 16:03:37.282269 16142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:37.285039 16212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:37.287004 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 16142
I20260806 16:03:37.287151 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:37.287461 25726 external_mini_cluster.cc:1428] Setting key 9b7723b6314aa4c2acfedabec669de31
I20260806 16:03:37.291765 16212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30: Bootstrap starting.
I20260806 16:03:37.292510 16215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:37.288822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52273 (local address 127.25.31.190:36955)
0806 16:03:37.289221 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:37.289236 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:37.289270 (+ 34us) server_negotiation.cc:407] Connection header received
0806 16:03:37.289905 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:37.289921 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:37.290186 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:37.290540 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:37.291346 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:37.291351 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:37.291363 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:37.291374 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:37.291427 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:37.291445 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:37.291558 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:37.291652 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 16:03:37.291810 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":129,"threads_started":1}
I20260806 16:03:37.294446 16212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:37.295300 16212 log.cc:824] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:37.295408 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:37.297243 16212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30: No bootstrap required, opened a new log
I20260806 16:03:37.299882 16215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:37.297373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47061 (local address 127.25.31.190:36955)
0806 16:03:37.297488 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:37.297493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:37.298592 (+ 1099us) server_negotiation.cc:407] Connection header received
0806 16:03:37.298698 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:37.298700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:37.298729 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:37.298798 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:37.299470 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:37.299472 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:37.299474 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 16:03:37.299484 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:37.299508 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 16:03:37.299517 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:37.299584 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:37.299689 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 16:03:37.299720 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 16:03:37.301465 16212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5054ce08ba12459d9c900d883b166c30" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 36955 } }
I20260806 16:03:37.301697 16212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 16:03:37.301754 16212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5054ce08ba12459d9c900d883b166c30, State: Initialized, Role: FOLLOWER
I20260806 16:03:37.302134 16212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5054ce08ba12459d9c900d883b166c30" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 36955 } }
I20260806 16:03:37.302286 16212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 16:03:37.302382 16212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 16:03:37.302477 16212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:37.303328 16212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5054ce08ba12459d9c900d883b166c30" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 36955 } }
I20260806 16:03:37.303654 16212 leader_election.cc:302] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5054ce08ba12459d9c900d883b166c30; no voters:
I20260806 16:03:37.303973 16212 leader_election.cc:288] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 16:03:37.304086 16222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:37.304322 16222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [term 1 LEADER]: Becoming Leader. State: Replica: 5054ce08ba12459d9c900d883b166c30, State: Running, Role: LEADER
I20260806 16:03:37.304692 16222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5054ce08ba12459d9c900d883b166c30" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 36955 } }
I20260806 16:03:37.305154 16212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:37.306438 16224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5054ce08ba12459d9c900d883b166c30. Latest consensus state: current_term: 1 leader_uuid: "5054ce08ba12459d9c900d883b166c30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5054ce08ba12459d9c900d883b166c30" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 36955 } } }
I20260806 16:03:37.306569 16224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:37.306871 16223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5054ce08ba12459d9c900d883b166c30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5054ce08ba12459d9c900d883b166c30" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 36955 } } }
I20260806 16:03:37.306948 16231 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:37.306978 16223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:37.309944 16231 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:37.315083 16231 catalog_manager.cc:1347] Generated new cluster ID: 40b753943b5a44d18729edf674ce9d9a
I20260806 16:03:37.315145 16231 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:37.327358 16231 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:37.327647 16231 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:37.337037 16231 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5054ce08ba12459d9c900d883b166c30: Generated new TSK 0
I20260806 16:03:37.337671 16231 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 16:03:37.344228 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 16142
2026-08-06T16:03:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1391 ms)
[----------] 18 tests from AuthTokenIssuingTest (25390 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:37 dist-test-slave-tm1g krb5kdc[16243](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:37 dist-test-slave-tm1g krb5kdc[16243](info): set up 2 sockets
Aug 06 16:03:37 dist-test-slave-tm1g krb5kdc[16243](info): commencing operation
krb5kdc: starting...
W20260806 16:03:38.390797 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:38 dist-test-slave-tm1g krb5kdc[16243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:03:38.509222 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:40481
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:42201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:40481,127.25.31.189:44617,127.25.31.188:46669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:38.621627 16256 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:03:38.621891 16256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:38.621915 16256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:38.622002 16256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:38.626114 16256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:38.626171 16256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:38.626190 16256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:38.626206 16256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:38.626247 16256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:38.631387 16256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:42201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:40481,127.25.31.189:44617,127.25.31.188:46669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:40481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:38.632407 16256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:38.633541 16256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:38.642517 16264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:38.642560 16262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:38.642560 16261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:38.642787 16256 server_base.cc:1034] running on GCE node
I20260806 16:03:38.643399 16256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:38.644479 16256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:38.645653 16256 hybrid_clock.cc:648] HybridClock initialized: now 1786032218645627 us; error 38 us; skew 500 ppm
Aug 06 16:03:38 dist-test-slave-tm1g krb5kdc[16243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032218, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:38.652771 16256 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:03:38.653687 16256 webserver.cc:460] Webserver started at http://127.25.31.190:42915/ using document root <none> and password file <none>
I20260806 16:03:38.654238 16256 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:38.654348 16256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:38.654588 16256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:38.656590 16256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "f1478d8372054f69840283866b02200a"
format_stamp: "Formatted at 2026-08-06 16:03:38 on dist-test-slave-tm1g"
server_key: "e0ba844288404e4141c04ec7386b7d55"
server_key_iv: "c8b1c7015d723e13bcbd19fd9ae9f9df"
server_key_version: "encryptionkey@0"
I20260806 16:03:38.657143 16256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "f1478d8372054f69840283866b02200a"
format_stamp: "Formatted at 2026-08-06 16:03:38 on dist-test-slave-tm1g"
server_key: "e0ba844288404e4141c04ec7386b7d55"
server_key_iv: "c8b1c7015d723e13bcbd19fd9ae9f9df"
server_key_version: "encryptionkey@0"
I20260806 16:03:38.661206 16256 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 16:03:38.663537 16272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:38.664723 16256 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:03:38.664870 16256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "f1478d8372054f69840283866b02200a"
format_stamp: "Formatted at 2026-08-06 16:03:38 on dist-test-slave-tm1g"
server_key: "e0ba844288404e4141c04ec7386b7d55"
server_key_iv: "c8b1c7015d723e13bcbd19fd9ae9f9df"
server_key_version: "encryptionkey@0"
I20260806 16:03:38.664975 16256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:38.700846 16256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:38.704276 16256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:38.704506 16256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:38.711836 16256 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:40481
I20260806 16:03:38.712127 16324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:40481 every 8 connection(s)
I20260806 16:03:38.712836 16256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:38.715571 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 16256
I20260806 16:03:38.715710 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:38.715713 16325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:38.716046 25726 external_mini_cluster.cc:1428] Setting key ca90ae68a26a646b6bea64ed1241577f
I20260806 16:03:38.720023 16325 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.31.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.189@KRBTEST.COM" created.
Aug 06 16:03:38 dist-test-slave-tm1g krb5kdc[16243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032218, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:03:38.735263 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.722542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40481 (local address 127.0.0.1:60050)
0806 16:03:38.723063 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:38.723107 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:38.723991 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:38.724680 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:38.724685 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:38.725112 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:38.725430 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:38.725446 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.726705 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.726708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:38.727542 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:38.727551 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.728181 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.728185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:38.728381 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:38.728977 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:38.728993 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:38.730399 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:38.732296 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:38.732304 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:38.732314 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:38.732537 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:38.732850 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:38.732854 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:38.732856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:38.732988 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:38.733573 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:38.733597 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:38.733730 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 16:03:38.733887 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 16:03:38.734259 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":144,"thread_start_us":136,"threads_started":1}
I20260806 16:03:38.735263 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.723529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60050 (local address 127.25.31.190:40481)
0806 16:03:38.723816 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:38.723827 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:38.723875 (+ 48us) server_negotiation.cc:407] Connection header received
0806 16:03:38.724278 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:38.724282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:38.724487 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:38.724681 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:38.725576 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.726579 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.727661 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.728032 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.728290 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:38.730543 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:38.730553 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:38.730562 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:38.730581 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:38.732145 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:38.732645 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:38.732650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:38.732652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:38.732757 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:38.733079 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:38.733084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:38.733086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:38.733444 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:38.733602 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:38.733971 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 16:03:38.734206 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":192,"thread_start_us":94,"threads_started":1}
I20260806 16:03:38.737810 16325 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:03:38.738655 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.738395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:44617 (local address 127.0.0.1:55010)
0806 16:03:38.738520 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:38.738594 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 16:03:38.738881 16275 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.31.189:44617: connect: Connection refused (error 111)
I20260806 16:03:38.739823 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.739581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:44617 (local address 127.0.0.1:55022)
0806 16:03:38.739683 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:38.739748 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 16:03:38.739967 16325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.189:44617: Network error: Client connection negotiation failed: client connection to 127.25.31.189:44617: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.189.keytab.
Entry for principal kudu/127.25.31.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.189.keytab.
I20260806 16:03:38.741602 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.189:44617
--webserver_interface=127.25.31.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.189.keytab
--principal=kudu/127.25.31.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:42201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:40481,127.25.31.189:44617,127.25.31.188:46669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:03:38.789259 16325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } attempt: 1
I20260806 16:03:38.790169 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.789857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:44617 (local address 127.0.0.1:55032)
0806 16:03:38.790016 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:38.790089 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 16:03:38.790938 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.790726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:44617 (local address 127.0.0.1:55048)
0806 16:03:38.790829 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:38.790876 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 16:03:38.791095 16325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.189:44617: Network error: Client connection negotiation failed: client connection to 127.25.31.189:44617: connect: Connection refused (error 111)
W20260806 16:03:38.854234 16332 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:03:38.854591 16332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:38.854655 16332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:38.854750 16332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:38.858745 16332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:38.858798 16332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:38.858819 16332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:38.858835 16332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:38.858850 16332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:38.859254 16325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } attempt: 2
I20260806 16:03:38.860111 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.859810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:44617 (local address 127.0.0.1:55064)
0806 16:03:38.859966 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:38.860037 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 16:03:38.860936 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.860687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:44617 (local address 127.0.0.1:55072)
0806 16:03:38.860811 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:38.860872 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:44617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 16:03:38.861161 16325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.189:44617: Network error: Client connection negotiation failed: client connection to 127.25.31.189:44617: connect: Connection refused (error 111)
I20260806 16:03:38.864590 16332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:42201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:40481,127.25.31.189:44617,127.25.31.188:46669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.189.keytab
--principal=kudu/127.25.31.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.189:44617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.31.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:38.865646 16332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:38.866734 16332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:38.875491 16337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:38.875475 16340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:38.875475 16338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:38.875742 16332 server_base.cc:1034] running on GCE node
I20260806 16:03:38.876387 16332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:38.877472 16332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:38.878665 16332 hybrid_clock.cc:648] HybridClock initialized: now 1786032218878638 us; error 50 us; skew 500 ppm
Aug 06 16:03:38 dist-test-slave-tm1g krb5kdc[16243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032218, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:38.881389 16332 init.cc:377] Logged in from keytab as kudu/127.25.31.189@KRBTEST.COM (short username kudu)
I20260806 16:03:38.882294 16332 webserver.cc:460] Webserver started at http://127.25.31.189:38449/ using document root <none> and password file <none>
I20260806 16:03:38.882907 16332 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:38.883000 16332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:38.883244 16332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:38.885278 16332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/data/instance:
uuid: "511680a5670f43c4852965d922c9efd5"
format_stamp: "Formatted at 2026-08-06 16:03:38 on dist-test-slave-tm1g"
server_key: "7162986f11b70707b4233f67e83c26d5"
server_key_iv: "a646fd5e6a8d955d244c85b48662ea6b"
server_key_version: "encryptionkey@0"
I20260806 16:03:38.885900 16332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/wal/instance:
uuid: "511680a5670f43c4852965d922c9efd5"
format_stamp: "Formatted at 2026-08-06 16:03:38 on dist-test-slave-tm1g"
server_key: "7162986f11b70707b4233f67e83c26d5"
server_key_iv: "a646fd5e6a8d955d244c85b48662ea6b"
server_key_version: "encryptionkey@0"
I20260806 16:03:38.889721 16332 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 16:03:38.892450 16348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:38.894258 16332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 16:03:38.894479 16332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/wal
uuid: "511680a5670f43c4852965d922c9efd5"
format_stamp: "Formatted at 2026-08-06 16:03:38 on dist-test-slave-tm1g"
server_key: "7162986f11b70707b4233f67e83c26d5"
server_key_iv: "a646fd5e6a8d955d244c85b48662ea6b"
server_key_version: "encryptionkey@0"
I20260806 16:03:38.894610 16332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:38.916508 16332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:38.920061 16332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:38.920289 16332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:38.927565 16332 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.189:44617
I20260806 16:03:38.927887 16400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.189:44617 every 8 connection(s)
I20260806 16:03:38.928606 16332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/data/info.pb
I20260806 16:03:38.931599 16401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:38.936115 16401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:03:38.938092 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 16332
I20260806 16:03:38.938241 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-1/wal/instance
I20260806 16:03:38.938549 25726 external_mini_cluster.cc:1428] Setting key 5b48b2453b9d2d2d9e09154dc2160cff
Aug 06 16:03:38 dist-test-slave-tm1g krb5kdc[16243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032218, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.189@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.31.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.188@KRBTEST.COM" created.
I20260806 16:03:38.951875 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.938092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60056 (local address 127.25.31.190:40481)
0806 16:03:38.938271 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:38.938277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:38.939176 (+ 899us) server_negotiation.cc:407] Connection header received
0806 16:03:38.940133 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:38.940137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:38.940197 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:38.940276 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:38.941565 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.942136 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.944067 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.944309 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.944407 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:38.947368 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:38.947372 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:38.947374 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:38.947401 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:38.949332 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:38.949911 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:38.949914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:38.949916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:38.950030 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:38.950532 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:38.950535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:38.950536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:38.950726 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:38.950810 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:38.951502 (+ 692us) server_negotiation.cc:299] Negotiation successful
0806 16:03:38.951652 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 16:03:38.952337 16325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } attempt: 3
I20260806 16:03:38.952848 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.938501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40481 (local address 127.0.0.1:60056)
0806 16:03:38.939001 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:38.939055 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:38.939915 (+ 860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:38.940448 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:38.940456 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:38.940789 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:38.941383 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:38.941406 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.942285 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.942292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:38.943902 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:38.943919 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.944489 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.944497 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:38.944755 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:38.945403 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:38.945424 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:38.947176 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:38.949446 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:38.949453 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:38.949467 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:38.949790 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:38.950127 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:38.950130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:38.950132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:38.950398 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:38.950840 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:38.950850 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:38.951203 (+ 353us) client_negotiation.cc:769] Sending connection context
0806 16:03:38.951414 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 16:03:38.951785 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":165,"threads_started":1}
I20260806 16:03:38.954480 16401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:03:38 dist-test-slave-tm1g krb5kdc[16243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032218, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for kudu/127.25.31.189@KRBTEST.COM
Aug 06 16:03:38 dist-test-slave-tm1g krb5kdc[16243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032218, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.189@KRBTEST.COM for kudu/127.25.31.189@KRBTEST.COM
I20260806 16:03:38.963442 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.952789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:44617 (local address 127.0.0.1:55078)
0806 16:03:38.952991 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:38.953008 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:38.953083 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:38.953921 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:38.953923 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:38.954112 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:38.954210 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:38.954215 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.955542 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.955548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:38.957065 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:38.957076 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.957699 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.957703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:38.957821 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:38.958284 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:38.958302 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:38.959533 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:38.961504 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:38.961508 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:38.961510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:38.961783 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:38.962119 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:38.962122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:38.962123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:38.962207 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:38.962976 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:38.962978 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:38.963060 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 16:03:38.963143 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 16:03:38.963318 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260806 16:03:38.963692 16406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.952890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55078 (local address 127.25.31.189:44617)
0806 16:03:38.953452 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:38.953463 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:38.953495 (+ 32us) server_negotiation.cc:407] Connection header received
0806 16:03:38.953550 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:38.953557 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:38.953774 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:38.953942 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:38.954387 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.955408 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.957191 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.957584 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.957700 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:38.959662 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:38.959672 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:38.959690 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:38.959711 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:38.961375 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:38.961904 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:38.961911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:38.961913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:38.962018 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:38.962423 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:38.962426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:38.962427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:38.962840 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:38.963001 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:38.963289 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 16:03:38.963469 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":462,"thread_start_us":84,"threads_started":1}
I20260806 16:03:38.965658 16325 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:03:38.966255 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.954975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:44617 (local address 127.0.0.1:55092)
0806 16:03:38.955155 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:38.955172 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:38.955248 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:38.955645 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:38.955647 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:38.955769 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:38.955890 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:38.955897 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.956610 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.956616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:38.957121 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:38.957128 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.957909 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.957916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:38.958012 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:38.958487 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:38.958508 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:38.961853 (+ 3345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:38.964659 (+ 2806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:38.964662 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:38.964664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:38.964910 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:38.965189 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:38.965192 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:38.965194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:38.965283 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:38.965785 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:38.965787 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:38.965873 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 16:03:38.965982 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 16:03:38.966103 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.188.keytab.
Entry for principal kudu/127.25.31.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.188.keytab.
I20260806 16:03:38.966370 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.955032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55092 (local address 127.25.31.189:44617)
0806 16:03:38.955362 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:38.955370 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:38.955382 (+ 12us) server_negotiation.cc:407] Connection header received
0806 16:03:38.955414 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:38.955417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:38.955462 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:38.955552 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:38.955992 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.956505 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.957245 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:38.957686 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:38.957802 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:38.961992 (+ 4190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:38.961996 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:38.961998 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:38.962018 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:38.964549 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:38.965005 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:38.965007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:38.965009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:38.965094 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:38.965411 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:38.965416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:38.965420 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:38.965642 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:38.965744 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:38.966035 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 16:03:38.966157 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":277,"thread_start_us":143,"threads_started":1}
I20260806 16:03:38.967016 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.966098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:46669 (local address 127.0.0.1:40108)
0806 16:03:38.966299 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:38.966754 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260806 16:03:38.967136 16401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:03:38.967195 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.188:46669
--webserver_interface=127.25.31.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.188.keytab
--principal=kudu/127.25.31.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:42201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:40481,127.25.31.189:44617,127.25.31.188:46669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:03:38.967820 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.967547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:46669 (local address 127.0.0.1:40126)
0806 16:03:38.967709 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:38.967770 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 16:03:38.967847 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.967495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:46669 (local address 127.0.0.1:40112)
0806 16:03:38.967715 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:38.967770 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260806 16:03:38.967976 16325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:46669: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
W20260806 16:03:38.968034 16349 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
I20260806 16:03:38.968881 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.968696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:46669 (local address 127.0.0.1:40132)
0806 16:03:38.968793 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:38.968835 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 16:03:38.969089 16401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:46669: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
I20260806 16:03:38.999166 16325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } attempt: 1
I20260806 16:03:39.000185 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:38.999801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:46669 (local address 127.0.0.1:40148)
0806 16:03:38.999993 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:39.000072 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 16:03:39.001196 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.000784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:46669 (local address 127.0.0.1:40160)
0806 16:03:39.001042 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:39.001110 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 16:03:39.001379 16325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:46669: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
I20260806 16:03:39.018390 16401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } attempt: 1
I20260806 16:03:39.019291 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.018981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:46669 (local address 127.0.0.1:40176)
0806 16:03:39.019141 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:39.019205 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 16:03:39.020191 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.019947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:46669 (local address 127.0.0.1:40180)
0806 16:03:39.020082 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:39.020133 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 16:03:39.020375 16401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:46669: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
I20260806 16:03:39.076558 16325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } attempt: 2
I20260806 16:03:39.077435 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.077086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:46669 (local address 127.0.0.1:40190)
0806 16:03:39.077258 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:39.077320 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 16:03:39.078351 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.078068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:46669 (local address 127.0.0.1:40202)
0806 16:03:39.078191 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:39.078243 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 16:03:39.078493 16325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:46669: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
W20260806 16:03:39.082417 16408 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:03:39.082719 16408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:39.082780 16408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:39.082870 16408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:39.086948 16408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:39.087019 16408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:39.087067 16408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:39.087122 16408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:39.087160 16408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:39.088546 16401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } attempt: 2
I20260806 16:03:39.089339 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.089052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:46669 (local address 127.0.0.1:40214)
0806 16:03:39.089191 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:39.089249 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 16:03:39.090037 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.089832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:46669 (local address 127.0.0.1:40218)
0806 16:03:39.089937 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:39.089989 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 16:03:39.090175 16401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:46669: Network error: Client connection negotiation failed: client connection to 127.25.31.188:46669: connect: Connection refused (error 111)
I20260806 16:03:39.092860 16408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:42201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:40481,127.25.31.189:44617,127.25.31.188:46669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.188.keytab
--principal=kudu/127.25.31.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.188:46669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.31.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:39.093923 16408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:39.094983 16408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:39.103303 16414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:39.103314 16415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:39.103500 16408 server_base.cc:1034] running on GCE node
W20260806 16:03:39.103312 16417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:39.104180 16408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:39.105321 16408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:39.106513 16408 hybrid_clock.cc:648] HybridClock initialized: now 1786032219106500 us; error 29 us; skew 500 ppm
Aug 06 16:03:39 dist-test-slave-tm1g krb5kdc[16243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032219, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:39.109265 16408 init.cc:377] Logged in from keytab as kudu/127.25.31.188@KRBTEST.COM (short username kudu)
I20260806 16:03:39.110131 16408 webserver.cc:460] Webserver started at http://127.25.31.188:39719/ using document root <none> and password file <none>
I20260806 16:03:39.110744 16408 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:39.110841 16408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:39.111088 16408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:39.113144 16408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/data/instance:
uuid: "d862c112b019481dba8e7ccb11a9d4a4"
format_stamp: "Formatted at 2026-08-06 16:03:39 on dist-test-slave-tm1g"
server_key: "7a3d6dd861793b339562da769a38564a"
server_key_iv: "ee501614c521493c446f3856c3caec71"
server_key_version: "encryptionkey@0"
I20260806 16:03:39.113724 16408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/wal/instance:
uuid: "d862c112b019481dba8e7ccb11a9d4a4"
format_stamp: "Formatted at 2026-08-06 16:03:39 on dist-test-slave-tm1g"
server_key: "7a3d6dd861793b339562da769a38564a"
server_key_iv: "ee501614c521493c446f3856c3caec71"
server_key_version: "encryptionkey@0"
I20260806 16:03:39.117424 16408 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 16:03:39.119875 16425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:39.121120 16408 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 16:03:39.121261 16408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/wal
uuid: "d862c112b019481dba8e7ccb11a9d4a4"
format_stamp: "Formatted at 2026-08-06 16:03:39 on dist-test-slave-tm1g"
server_key: "7a3d6dd861793b339562da769a38564a"
server_key_iv: "ee501614c521493c446f3856c3caec71"
server_key_version: "encryptionkey@0"
I20260806 16:03:39.121373 16408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:39.143539 16408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:39.146713 16408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:39.146920 16408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:39.153673 16408 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.188:46669
I20260806 16:03:39.153964 16477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.188:46669 every 8 connection(s)
I20260806 16:03:39.154707 16408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/data/info.pb
I20260806 16:03:39.157369 16478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:39.161037 16478 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:03:39.163947 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 16408
I20260806 16:03:39.164063 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786031939584994-25726-0/minicluster-data/master-2/wal/instance
I20260806 16:03:39.164336 25726 external_mini_cluster.cc:1428] Setting key 501747f24b531119bf48f05cb0127c60
Aug 06 16:03:39 dist-test-slave-tm1g krb5kdc[16243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032219, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.188@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
Aug 06 16:03:39 dist-test-slave-tm1g krb5kdc[16243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.188@KRBTEST.COM
I20260806 16:03:39.177631 16325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } attempt: 3
I20260806 16:03:39.178575 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.162870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60058 (local address 127.25.31.190:40481)
0806 16:03:39.163017 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.163023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.163814 (+ 791us) server_negotiation.cc:407] Connection header received
0806 16:03:39.164864 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.164867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.164926 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.165015 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.166174 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.166761 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.167840 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.168066 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.168171 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.173895 (+ 5724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:39.173898 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:39.173900 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:39.173923 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:39.175630 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.176291 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.176295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.176296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.176376 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.176758 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.176764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.176769 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.177023 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:39.177128 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:39.177641 (+ 513us) server_negotiation.cc:299] Negotiation successful
0806 16:03:39.177762 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 16:03:39.179060 16481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.165012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40228 (local address 127.25.31.188:46669)
0806 16:03:39.165278 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.165291 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.165548 (+ 257us) server_negotiation.cc:407] Connection header received
0806 16:03:39.165700 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.165710 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.165863 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.166024 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.166869 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.167752 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.168751 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.169167 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.169246 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.174266 (+ 5020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:39.174271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:39.174281 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:39.174345 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:39.176151 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.176610 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.176615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.176616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.176687 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.176972 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.176975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.176977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.177400 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:39.177579 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:39.178127 (+ 548us) server_negotiation.cc:299] Negotiation successful
0806 16:03:39.178270 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":169,"spinlock_wait_cycles":4352,"thread_start_us":101,"threads_started":1}
I20260806 16:03:39.179060 16480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.163242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:40481 (local address 127.0.0.1:60058)
0806 16:03:39.163663 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:39.163704 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:39.164641 (+ 937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:39.165217 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:39.165227 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:39.165589 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:39.166030 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:39.166042 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.166873 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.166876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:39.167708 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:39.167716 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.168217 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.168220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:39.168389 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.172141 (+ 3752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:39.172162 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:39.173684 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:39.175752 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:39.175760 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:39.175768 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:39.176162 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:39.176475 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:39.176478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:39.176480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:39.176664 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:39.177133 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:39.177138 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:39.177382 (+ 244us) client_negotiation.cc:769] Sending connection context
0806 16:03:39.177572 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 16:03:39.177888 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":2,"spinlock_wait_cycles":3456,"thread_start_us":111,"threads_started":1}
I20260806 16:03:39.180678 16478 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:03:39.181315 16401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } attempt: 3
Aug 06 16:03:39 dist-test-slave-tm1g krb5kdc[16243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032218, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for kudu/127.25.31.188@KRBTEST.COM
I20260806 16:03:39.187552 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.178072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:46669 (local address 127.0.0.1:40236)
0806 16:03:39.178251 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:39.178263 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:39.178377 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:39.179060 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:39.179063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:39.179257 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:39.179382 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:39.179388 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.180351 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.180354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:39.181130 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:39.181140 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.181738 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.181742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:39.181836 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.182340 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:39.182356 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:39.183907 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:39.185898 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:39.185902 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:39.185907 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:39.186152 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:39.186463 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:39.186466 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:39.186468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:39.186558 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:39.187023 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:39.187027 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:39.187121 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 16:03:39.187230 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 16:03:39.187360 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
Aug 06 16:03:39 dist-test-slave-tm1g krb5kdc[16243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032219, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.188@KRBTEST.COM for kudu/127.25.31.189@KRBTEST.COM
I20260806 16:03:39.187995 16483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.178245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40236 (local address 127.25.31.188:46669)
0806 16:03:39.178852 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.178862 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.178882 (+ 20us) server_negotiation.cc:407] Connection header received
0806 16:03:39.178910 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.178912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.178945 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.179044 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.179500 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.180239 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.181260 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.181618 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.181735 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.184047 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:39.184051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:39.184053 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:39.184075 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:39.185776 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.186264 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.186268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.186272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.186385 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.186674 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.186679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.186684 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.186897 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:39.187009 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:39.187767 (+ 758us) server_negotiation.cc:299] Negotiation successful
0806 16:03:39.187877 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":166,"threads_started":1}
Aug 06 16:03:39 dist-test-slave-tm1g krb5kdc[16243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032218, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.189@KRBTEST.COM for kudu/127.25.31.188@KRBTEST.COM
Aug 06 16:03:39 dist-test-slave-tm1g krb5kdc[16243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:03:39.191406 16325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a: Bootstrap starting.
I20260806 16:03:39.192565 16403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.181696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:46669 (local address 127.0.0.1:40240)
0806 16:03:39.181958 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:39.181975 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:39.182349 (+ 374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:39.182681 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:39.182684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:39.182865 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:39.182962 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:39.182969 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.184785 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.184789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:39.185829 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:39.185839 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.186477 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.186480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:39.186564 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.187096 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:39.187110 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:39.188937 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:39.190948 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:39.190951 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:39.190953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:39.191194 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:39.191519 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:39.191522 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:39.191524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:39.191614 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:39.192057 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:39.192060 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:39.192149 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 16:03:39.192289 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 16:03:39.192416 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":88}
I20260806 16:03:39.193048 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.181311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55102 (local address 127.25.31.189:44617)
0806 16:03:39.181555 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.181560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.181575 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:03:39.181618 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.181622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.181674 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.181747 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.182545 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.183288 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.184465 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.184839 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.184918 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.189039 (+ 4121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:39.189043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:39.189046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:39.189071 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:39.191024 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.191572 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.191576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.191579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.191682 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.192032 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.192037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.192041 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.192263 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:39.192342 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:39.192739 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 16:03:39.192860 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":223,"server-negotiator.queue_time_us":153}
I20260806 16:03:39.193220 16481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.181934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40240 (local address 127.25.31.188:46669)
0806 16:03:39.182049 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.182052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.182065 (+ 13us) server_negotiation.cc:407] Connection header received
0806 16:03:39.182521 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.182527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.182568 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.183069 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.183142 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.184671 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.185937 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.186325 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.186761 (+ 436us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.189090 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:39.189093 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:39.189095 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:39.189121 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:39.190806 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.191319 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.191322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.191324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.191406 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.191728 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.191731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.191733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.191941 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:39.192884 (+ 943us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:39.192960 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 16:03:39.193064 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44,"spinlock_wait_cycles":4480}
I20260806 16:03:39.193589 16480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.181195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:44617 (local address 127.0.0.1:55102)
0806 16:03:39.181443 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:39.181457 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:39.181534 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:39.182065 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:39.182069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:39.182242 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:39.182401 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:39.182412 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.183448 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.183452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:39.184308 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:39.184321 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.185009 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.185014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:39.185130 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.186736 (+ 1606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:39.186752 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:39.188869 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:39.191157 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:39.191162 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:39.191166 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:39.191441 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:39.191808 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:39.191813 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:39.191816 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:39.191911 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:39.192436 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:39.192440 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:39.192534 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 16:03:39.193241 (+ 707us) client_negotiation.cc:240] Negotiation successful
0806 16:03:39.193390 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260806 16:03:39.194363 16478 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:03:39.194571 16325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:39.195484 16325 log.cc:824] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:39.195688 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.182290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60072 (local address 127.25.31.190:40481)
0806 16:03:39.182412 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.182416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.182584 (+ 168us) server_negotiation.cc:407] Connection header received
0806 16:03:39.182745 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.182748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.182787 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.182838 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.183709 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.184419 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.185617 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.185960 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.186061 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.190682 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:39.190687 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:39.190690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:39.190719 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:39.192490 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.193562 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.193565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.193567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.194099 (+ 532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.194520 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.194524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.194525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.194813 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:39.194998 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:39.195393 (+ 395us) server_negotiation.cc:299] Negotiation successful
0806 16:03:39.195517 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:03:39.196250 16401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5: Bootstrap starting.
I20260806 16:03:39.197568 16325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a: No bootstrap required, opened a new log
I20260806 16:03:39.198765 16401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:39.199605 16401 log.cc:824] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:39.201810 16401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5: No bootstrap required, opened a new log
I20260806 16:03:39.201803 16325 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } }
Aug 06 16:03:39 dist-test-slave-tm1g krb5kdc[16243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.189@KRBTEST.COM
I20260806 16:03:39.202008 16325 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1478d8372054f69840283866b02200a, State: Initialized, Role: FOLLOWER
I20260806 16:03:39.202399 16325 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } }
I20260806 16:03:39.203820 16325 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 16:03:39 dist-test-slave-tm1g krb5kdc[16243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032219, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.188@KRBTEST.COM for kudu/127.25.31.188@KRBTEST.COM
I20260806 16:03:39.206907 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.196847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55118 (local address 127.25.31.189:44617)
0806 16:03:39.196970 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.196974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.196985 (+ 11us) server_negotiation.cc:407] Connection header received
0806 16:03:39.197021 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.197023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.197056 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.197140 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.197828 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.198502 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.199408 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.199642 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.199757 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.202833 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:39.202836 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:39.202839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:39.202863 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:39.204789 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.205343 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.205347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.205349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.205441 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.205749 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.205755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.205758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.205968 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:39.206182 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:39.206656 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 16:03:39.206769 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 16:03:39.206184 16278 tablet.cc:2369] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 16:03:39.208777 16480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.194821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:46669 (local address 127.0.0.1:40246)
0806 16:03:39.195135 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:39.195148 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:39.195216 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:39.200362 (+ 5146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:39.200364 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:39.200532 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:39.200616 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:39.200622 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.201549 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.201552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:39.202283 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:39.202293 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.202978 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.202981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:39.203047 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.203423 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:39.203438 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:39.204938 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:39.207358 (+ 2420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:39.207361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:39.207363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:39.207585 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:39.207843 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:39.207846 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:39.207848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:39.207935 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:39.208323 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:39.208326 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:39.208417 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 16:03:39.208514 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 16:03:39.208635 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257}
I20260806 16:03:39.209594 16481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.195034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40246 (local address 127.25.31.188:46669)
0806 16:03:39.200102 (+ 5068us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.200106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.200120 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:03:39.200155 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.200157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.200196 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.200272 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.200728 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.201437 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.202441 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.202779 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.202882 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.205071 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:39.205076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:39.205081 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:39.205107 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:39.207253 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.207685 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.207688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.207689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.207762 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.208019 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.208022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.208023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.208198 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:39.208287 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:39.209366 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0806 16:03:39.209468 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4996}
W20260806 16:03:39.211248 16498 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:03:39.211324 16498 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:03:39.209022 16256 master_runner.cc:186] Error getting master registration for 127.25.31.189:44617: OK, instance_id { permanent_uuid: "511680a5670f43c4852965d922c9efd5" instance_seqno: 1786032218925445 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:03:39.212445 16256 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:03:39.212509 16478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4: Bootstrap starting.
I20260806 16:03:39.212529 16401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } }
I20260806 16:03:39.212743 16401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 511680a5670f43c4852965d922c9efd5, State: Initialized, Role: FOLLOWER
I20260806 16:03:39.213060 16401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } }
I20260806 16:03:39.213547 16500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } } }
I20260806 16:03:39.213691 16500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:03:39.214365 16401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:39.215574 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:39.215853 16478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:39.216831 16478 log.cc:824] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:39.216914 16299 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f1478d8372054f69840283866b02200a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60072
I20260806 16:03:39.217123 16299 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 16:03:39.217236 16299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:39.218655 16299 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } }
I20260806 16:03:39.219619 16299 leader_election.cc:288] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [CANDIDATE]: Term 1 election: Requested vote from peers 511680a5670f43c4852965d922c9efd5 (127.25.31.189:44617), d862c112b019481dba8e7ccb11a9d4a4 (127.25.31.188:46669)
I20260806 16:03:39.220782 16478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4: No bootstrap required, opened a new log
I20260806 16:03:39.220712 16375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f1478d8372054f69840283866b02200a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "511680a5670f43c4852965d922c9efd5"
I20260806 16:03:39.220942 16375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:39.222222 16375 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1478d8372054f69840283866b02200a in term 1.
I20260806 16:03:39.222640 16275 leader_election.cc:302] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 511680a5670f43c4852965d922c9efd5, f1478d8372054f69840283866b02200a; no voters:
I20260806 16:03:39.223121 16487 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } } }
I20260806 16:03:39.223311 16487 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:03:39.223450 16487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:39.223546 16487 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [term 1 LEADER]: Becoming Leader. State: Replica: f1478d8372054f69840283866b02200a, State: Running, Role: LEADER
I20260806 16:03:39.224028 16487 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } }
W20260806 16:03:39.224177 16511 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:03:39.224241 16511 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:03:39.225116 16478 raft_consensus.cc:348] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } }
I20260806 16:03:39.225617 16478 raft_consensus.cc:729] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d862c112b019481dba8e7ccb11a9d4a4, State: Initialized, Role: FOLLOWER
I20260806 16:03:39.226091 16478 consensus_queue.cc:260] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } }
I20260806 16:03:39.227492 16478 sys_catalog.cc:564] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:39.229671 16452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f1478d8372054f69840283866b02200a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d862c112b019481dba8e7ccb11a9d4a4"
I20260806 16:03:39.230104 16518 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1478d8372054f69840283866b02200a. Latest consensus state: current_term: 1 leader_uuid: "f1478d8372054f69840283866b02200a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } } }
I20260806 16:03:39.230218 16518 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [sys.catalog]: This master's current role is: LEADER
W20260806 16:03:39.232586 16534 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:03:39.232712 16534 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:03:39.229907 16452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:39.235879 16452 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1478d8372054f69840283866b02200a in term 1.
I20260806 16:03:39.231528 16332 master_runner.cc:186] Error getting master registration for 127.25.31.188:46669: OK, instance_id { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" instance_seqno: 1786032219151768 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:03:39.238230 16332 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:03:39.238545 16535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:39.239480 16535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 16:03:39.239667 16520 sys_catalog.cc:455] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } } }
I20260806 16:03:39.240073 16520 sys_catalog.cc:458] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:03:39.245904 16452 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4 [term 1 FOLLOWER]: Refusing update from remote peer f1478d8372054f69840283866b02200a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:39.246529 16487 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:03:39.250864 16375 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5 [term 1 FOLLOWER]: Refusing update from remote peer f1478d8372054f69840283866b02200a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:39.251185 16520 sys_catalog.cc:455] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1478d8372054f69840283866b02200a. Latest consensus state: current_term: 1 leader_uuid: "f1478d8372054f69840283866b02200a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } } }
I20260806 16:03:39.251346 16520 sys_catalog.cc:458] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:03:39.251398 16487 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [LEADER]: Connected to new peer: Peer: permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:03:39.255086 16500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1478d8372054f69840283866b02200a. Latest consensus state: current_term: 1 leader_uuid: "f1478d8372054f69840283866b02200a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } } }
I20260806 16:03:39.255208 16500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:03:39.261526 16481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.224815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40256 (local address 127.25.31.188:46669)
0806 16:03:39.224932 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.224936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.239910 (+ 14974us) server_negotiation.cc:407] Connection header received
0806 16:03:39.240215 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.240219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.240263 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.240313 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.242290 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.243078 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.245497 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.245847 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.245917 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.256046 (+ 10129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:39.256050 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:39.256053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:39.256081 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:39.259309 (+ 3228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.259822 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.259825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.259827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.259912 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.260225 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.260229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.260230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.260463 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:39.260593 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:39.261240 (+ 647us) server_negotiation.cc:299] Negotiation successful
0806 16:03:39.261351 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 16:03:39.262838 16487 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1478d8372054f69840283866b02200a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } } }
I20260806 16:03:39.262987 16487 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:39.263116 16537 mvcc.cc:205] Tried to move back new op lower bound from 7315587970019684352 to 7315587969947803648. Current Snapshot: MvccSnapshot[applied={T|T < 7315587970019684352}]
I20260806 16:03:39.264721 16538 mvcc.cc:205] Tried to move back new op lower bound from 7315587970019684352 to 7315587969947803648. Current Snapshot: MvccSnapshot[applied={T|T < 7315587970019684352}]
I20260806 16:03:39.264880 16329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.225260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60080 (local address 127.25.31.190:40481)
0806 16:03:39.225384 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.225389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.233387 (+ 7998us) server_negotiation.cc:407] Connection header received
0806 16:03:39.233601 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.233605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.233652 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.233703 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.234813 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.235610 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.241313 (+ 5703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.241679 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.241753 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.255476 (+ 13723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:39.255480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:39.255483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:39.255511 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:39.258220 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.263447 (+ 5227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.263454 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.263459 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.263555 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.263883 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.263886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.263888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.264114 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:39.264240 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:39.264598 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 16:03:39.264732 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 16:03:39.266484 16520 sys_catalog.cc:455] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1478d8372054f69840283866b02200a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } } }
I20260806 16:03:39.266655 16520 sys_catalog.cc:458] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:03:39.272367 16407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:39.225597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55130 (local address 127.25.31.189:44617)
0806 16:03:39.225710 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.225714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.244389 (+ 18675us) server_negotiation.cc:407] Connection header received
0806 16:03:39.244568 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.244571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.244620 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.244668 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.247309 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.248097 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.255355 (+ 7258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:39.255721 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:39.258412 (+ 2691us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:39.268611 (+ 10199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:39.268615 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:39.268618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:39.268646 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:39.270572 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.271055 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.271059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.271061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.271143 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:39.271444 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:39.271447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:39.271449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:39.271658 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:39.271763 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:39.272098 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 16:03:39.272217 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 16:03:39.279649 16535 catalog_manager.cc:1347] Generated new cluster ID: 90054ad7a6cb4433badb1d91e8c5e0e8
I20260806 16:03:39.279714 16535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:39.282960 16518 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1478d8372054f69840283866b02200a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } } }
I20260806 16:03:39.283069 16518 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:39.284080 16500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1478d8372054f69840283866b02200a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1478d8372054f69840283866b02200a" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 40481 } } peers { permanent_uuid: "511680a5670f43c4852965d922c9efd5" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 44617 } } peers { permanent_uuid: "d862c112b019481dba8e7ccb11a9d4a4" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 46669 } } }
I20260806 16:03:39.284171 16500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:03:39.284591 16540 mvcc.cc:205] Tried to move back new op lower bound from 7315587970019684352 to 7315587969947803648. Current Snapshot: MvccSnapshot[applied={T|T < 7315587970019684352}]
I20260806 16:03:39.298782 16535 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:39.300143 16535 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:39.322054 16535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1478d8372054f69840283866b02200a: Generated new TSK 0
I20260806 16:03:39.323055 16535 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:03:39.381130 16329 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:39.379228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60086 (local address 127.25.31.190:40481)
0806 16:03:39.379459 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.379465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.379963 (+ 498us) server_negotiation.cc:407] Connection header received
0806 16:03:39.380178 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.380182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.380236 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.380297 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.380979 (+ 682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60086: BlockingRecv error: recv got EOF from 127.0.0.1:60086 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
W20260806 16:03:39.382633 16481 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:39.378791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40272 (local address 127.25.31.188:46669)
0806 16:03:39.379130 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.379136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.381690 (+ 2554us) server_negotiation.cc:407] Connection header received
0806 16:03:39.381860 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.381863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.381911 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.381964 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.382514 (+ 550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40272: BlockingRecv error: recv got EOF from 127.0.0.1:40272 (error 108)
Metrics: {"server-negotiator.queue_time_us":227}
W20260806 16:03:39.395447 16407 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:39.379463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55140 (local address 127.25.31.189:44617)
0806 16:03:39.379592 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.379597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.383022 (+ 3425us) server_negotiation.cc:407] Connection header received
0806 16:03:39.394482 (+ 11460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.394487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.394555 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.394621 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.395308 (+ 687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55140: BlockingRecv error: recv got EOF from 127.0.0.1:55140 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 16:03:39.404292 16407 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:39.402880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55144 (local address 127.25.31.189:44617)
0806 16:03:39.403001 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.403005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.403388 (+ 383us) server_negotiation.cc:407] Connection header received
0806 16:03:39.403580 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.403583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.403655 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.403740 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.404189 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55144: BlockingRecv error: recv got EOF from 127.0.0.1:55144 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 16:03:39.415588 16407 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:39.413689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55160 (local address 127.25.31.189:44617)
0806 16:03:39.413826 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.413831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.414475 (+ 644us) server_negotiation.cc:407] Connection header received
0806 16:03:39.414684 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.414688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.414745 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.414820 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.415476 (+ 656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55160: BlockingRecv error: recv got EOF from 127.0.0.1:55160 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 16:03:39.421679 16407 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:39.420037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55168 (local address 127.25.31.189:44617)
0806 16:03:39.420139 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.420143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.420719 (+ 576us) server_negotiation.cc:407] Connection header received
0806 16:03:39.420926 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.420929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.420982 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.421039 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.421578 (+ 539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55168: BlockingRecv error: recv got EOF from 127.0.0.1:55168 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 16:03:39.436139 16407 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:39.434841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55174 (local address 127.25.31.189:44617)
0806 16:03:39.434977 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.434982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.435257 (+ 275us) server_negotiation.cc:407] Connection header received
0806 16:03:39.435380 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.435384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.435444 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.435537 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.436031 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55174: BlockingRecv error: recv got EOF from 127.0.0.1:55174 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 16:03:39.447082 16407 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:39.445858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55184 (local address 127.25.31.189:44617)
0806 16:03:39.445965 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.445969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.446191 (+ 222us) server_negotiation.cc:407] Connection header received
0806 16:03:39.446358 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.446364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.446439 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.446528 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.446975 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55184: BlockingRecv error: recv got EOF from 127.0.0.1:55184 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 16:03:39.466379 16407 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:39.465318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55190 (local address 127.25.31.189:44617)
0806 16:03:39.465465 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.465469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.465602 (+ 133us) server_negotiation.cc:407] Connection header received
0806 16:03:39.465702 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.465704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.465750 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.465825 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.466247 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55190: BlockingRecv error: recv got EOF from 127.0.0.1:55190 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 16:03:39.501772 16407 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:39.500617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55200 (local address 127.25.31.189:44617)
0806 16:03:39.500775 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.500779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.500852 (+ 73us) server_negotiation.cc:407] Connection header received
0806 16:03:39.500975 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.500978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.501026 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.501099 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.501644 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55200: BlockingRecv error: recv got EOF from 127.0.0.1:55200 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 16:03:39.569098 16407 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:39.567892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55210 (local address 127.25.31.189:44617)
0806 16:03:39.568028 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.568031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.568157 (+ 126us) server_negotiation.cc:407] Connection header received
0806 16:03:39.568284 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.568288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.568334 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.568417 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.568998 (+ 581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55210: BlockingRecv error: recv got EOF from 127.0.0.1:55210 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 16:03:39.700580 16407 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:39.699430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55218 (local address 127.25.31.189:44617)
0806 16:03:39.699619 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.699623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.699707 (+ 84us) server_negotiation.cc:407] Connection header received
0806 16:03:39.699832 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.699835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.699884 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.699957 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.700425 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55218: BlockingRecv error: recv got EOF from 127.0.0.1:55218 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260806 16:03:39.959683 16407 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:39.958674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55224 (local address 127.25.31.189:44617)
0806 16:03:39.958814 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:39.958818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:39.958906 (+ 88us) server_negotiation.cc:407] Connection header received
0806 16:03:39.959033 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:39.959035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:39.959083 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:39.959166 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:39.959589 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55224: BlockingRecv error: recv got EOF from 127.0.0.1:55224 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:03:40.225679 16511 catalog_manager.cc:1259] Loaded cluster ID: 90054ad7a6cb4433badb1d91e8c5e0e8
I20260806 16:03:40.225769 16511 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5: loading cluster ID for follower catalog manager: success
I20260806 16:03:40.227730 16511 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5: acquiring CA information for follower catalog manager: success
I20260806 16:03:40.228936 16511 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 511680a5670f43c4852965d922c9efd5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:03:40.233774 16534 catalog_manager.cc:1259] Loaded cluster ID: 90054ad7a6cb4433badb1d91e8c5e0e8
I20260806 16:03:40.233834 16534 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4: loading cluster ID for follower catalog manager: success
I20260806 16:03:40.235401 16534 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4: acquiring CA information for follower catalog manager: success
I20260806 16:03:40.236537 16534 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d862c112b019481dba8e7ccb11a9d4a4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:03:40.477824 16610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:40.473984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55230 (local address 127.25.31.189:44617)
0806 16:03:40.474218 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:40.474222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:40.474353 (+ 131us) server_negotiation.cc:407] Connection header received
0806 16:03:40.474456 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:40.474458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:40.474591 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:40.474699 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:40.475056 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:40.475589 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:40.476399 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:40.476623 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:40.476697 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:40.477022 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:40.477451 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:40.477540 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:40.477651 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 16:03:40.477701 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":90,"threads_started":1}
I20260806 16:03:40.484457 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:40.480870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40278 (local address 127.25.31.188:46669)
0806 16:03:40.481115 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:40.481118 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:40.481132 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:03:40.481229 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:40.481231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:40.481334 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:40.481423 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:40.481650 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:40.482148 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:40.483055 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:40.483272 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:40.483350 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:40.483601 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:40.483964 (+ 363us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:40.484049 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:40.484228 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 16:03:40.484288 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":99,"threads_started":1}
I20260806 16:03:40.491626 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:40.487617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40290 (local address 127.25.31.188:46669)
0806 16:03:40.487731 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:40.487735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:40.487964 (+ 229us) server_negotiation.cc:407] Connection header received
0806 16:03:40.488106 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:40.488109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:40.488164 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:40.488231 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:40.488600 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:40.489228 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:40.490382 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:40.490711 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:40.490806 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:40.491151 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:40.491220 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:40.491264 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:40.491421 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 16:03:40.491476 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 16:03:40.491851 16610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:40.487452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55242 (local address 127.25.31.189:44617)
0806 16:03:40.487577 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:40.487582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:40.488301 (+ 719us) server_negotiation.cc:407] Connection header received
0806 16:03:40.488536 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:40.488539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:40.488592 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:40.488667 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:40.488912 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:40.489622 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:40.490689 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:40.491023 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:40.491137 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:40.491379 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:40.491459 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:40.491527 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:40.491668 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 16:03:40.491726 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 16:03:40.492247 16622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:40.487424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60094 (local address 127.25.31.190:40481)
0806 16:03:40.487878 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:40.487885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:40.487902 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:03:40.487991 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:40.487994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:40.488096 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:40.488219 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:40.488459 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:40.489237 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:40.490249 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:40.490605 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:40.490698 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:40.491025 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:40.491335 (+ 310us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:40.491421 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:40.492046 (+ 625us) server_negotiation.cc:299] Negotiation successful
0806 16:03:40.492109 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":70,"threads_started":1}
I20260806 16:03:40.495483 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 16256
I20260806 16:03:40.503011 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 16332
I20260806 16:03:40.509361 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 16408
2026-08-06T16:03:40Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3165 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 16:03:40 dist-test-slave-tm1g krb5kdc[16628](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 16:03:40 dist-test-slave-tm1g krb5kdc[16628](info): set up 2 sockets
Aug 06 16:03:40 dist-test-slave-tm1g krb5kdc[16628](info): commencing operation
krb5kdc: starting...
W20260806 16:03:41.556013 25726 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/test-user.keytab.
Aug 06 16:03:41 dist-test-slave-tm1g krb5kdc[16628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T16:03:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T16:03:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.31.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
Entry for principal kudu/127.25.31.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab.
I20260806 16:03:41.674232 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:34563
--webserver_interface=127.25.31.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:38717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:34563,127.25.31.189:33911,127.25.31.188:35493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 16:03:41.785578 16641 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:03:41.785892 16641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:41.785965 16641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:41.786058 16641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 16:03:41.790009 16641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:41.790096 16641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:41.790136 16641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:41.790175 16641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:41.790211 16641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:41.795338 16641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:38717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:34563,127.25.31.189:33911,127.25.31.188:35493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.190.keytab
--principal=kudu/127.25.31.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.190:34563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.31.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:41.796375 16641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:41.797390 16641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:41.805670 16649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:41.805667 16647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:41.805667 16646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:41.805910 16641 server_base.cc:1034] running on GCE node
I20260806 16:03:41.806577 16641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:41.807565 16641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:41.808779 16641 hybrid_clock.cc:648] HybridClock initialized: now 1786032221808753 us; error 41 us; skew 500 ppm
Aug 06 16:03:41 dist-test-slave-tm1g krb5kdc[16628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032221, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:41.811600 16641 init.cc:377] Logged in from keytab as kudu/127.25.31.190@KRBTEST.COM (short username kudu)
I20260806 16:03:41.812424 16641 webserver.cc:460] Webserver started at http://127.25.31.190:37417/ using document root <none> and password file <none>
I20260806 16:03:41.812961 16641 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:41.813050 16641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:41.813277 16641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:41.815328 16641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/data/instance:
uuid: "8dbcb460d1ce41a49a474b474c456176"
format_stamp: "Formatted at 2026-08-06 16:03:41 on dist-test-slave-tm1g"
server_key: "e21965de30589b2eec15def6983d1d86"
server_key_iv: "bd43d285c437e7f7ff2b23fa871f64b1"
server_key_version: "encryptionkey@0"
I20260806 16:03:41.815896 16641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/wal/instance:
uuid: "8dbcb460d1ce41a49a474b474c456176"
format_stamp: "Formatted at 2026-08-06 16:03:41 on dist-test-slave-tm1g"
server_key: "e21965de30589b2eec15def6983d1d86"
server_key_iv: "bd43d285c437e7f7ff2b23fa871f64b1"
server_key_version: "encryptionkey@0"
I20260806 16:03:41.819453 16641 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 16:03:41.821748 16656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:41.822988 16641 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 16:03:41.823127 16641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/wal
uuid: "8dbcb460d1ce41a49a474b474c456176"
format_stamp: "Formatted at 2026-08-06 16:03:41 on dist-test-slave-tm1g"
server_key: "e21965de30589b2eec15def6983d1d86"
server_key_iv: "bd43d285c437e7f7ff2b23fa871f64b1"
server_key_version: "encryptionkey@0"
I20260806 16:03:41.823239 16641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:41.841748 16641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:41.844436 16641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:41.844583 16641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:41.851537 16641 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.190:34563
I20260806 16:03:41.851837 16708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.190:34563 every 8 connection(s)
I20260806 16:03:41.852514 16641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/data/info.pb
I20260806 16:03:41.855252 16709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:41.858805 16709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:03:41.859972 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 16641
I20260806 16:03:41.860101 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-0/wal/instance
I20260806 16:03:41.860386 25726 external_mini_cluster.cc:1428] Setting key c8334ff41a72b104c63ff4dcb21737ac
Aug 06 16:03:41 dist-test-slave-tm1g krb5kdc[16628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032221, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.31.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.189@KRBTEST.COM" created.
I20260806 16:03:41.874745 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:41.861071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33874 (local address 127.25.31.190:34563)
0806 16:03:41.862274 (+ 1203us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:41.862283 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:41.862383 (+ 100us) server_negotiation.cc:407] Connection header received
0806 16:03:41.863099 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:41.863103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:41.863278 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:41.863426 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:41.864229 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:41.865731 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:41.866762 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:41.867411 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:41.867588 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:41.870042 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:41.870047 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:41.870056 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:41.870077 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:41.871704 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:41.872260 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:41.872266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:41.872268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:41.872383 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:41.872713 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:41.872719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:41.872724 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:41.873225 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:41.873388 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:41.873710 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 16:03:41.873937 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1105,"spinlock_wait_cycles":11008,"thread_start_us":147,"threads_started":1}
I20260806 16:03:41.874754 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:41.861071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:34563 (local address 127.0.0.1:33874)
0806 16:03:41.861522 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:41.861572 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:41.862786 (+ 1214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:41.863467 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:41.863471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:41.863812 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:41.864068 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:41.864081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:41.865850 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:41.865853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:41.866635 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:41.866644 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:41.867570 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:41.867573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:41.867664 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:41.868258 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:41.868273 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:41.869898 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:41.871810 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:41.871818 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:41.871824 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:41.872149 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:41.872500 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:41.872503 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:41.872504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:41.872613 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:41.873343 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:41.873348 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:41.873463 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 16:03:41.873594 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 16:03:41.874060 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":119,"thread_start_us":138,"threads_started":1}
I20260806 16:03:41.877242 16709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:03:41.877961 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:41.877719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:33911 (local address 127.0.0.1:32978)
0806 16:03:41.877830 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:41.877912 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:33911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 16:03:41.878149 16660 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.31.189:33911: connect: Connection refused (error 111)
I20260806 16:03:41.878962 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:41.878756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:33911 (local address 127.0.0.1:32992)
0806 16:03:41.878857 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:41.878911 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:33911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 16:03:41.879114 16709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.189:33911: Network error: Client connection negotiation failed: client connection to 127.25.31.189:33911: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.189.keytab.
Entry for principal kudu/127.25.31.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.189.keytab.
I20260806 16:03:41.889133 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.189:33911
--webserver_interface=127.25.31.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.189.keytab
--principal=kudu/127.25.31.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:38717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:34563,127.25.31.189:33911,127.25.31.188:35493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:03:41.928356 16709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } attempt: 1
I20260806 16:03:41.929262 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:41.928922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:33911 (local address 127.0.0.1:32994)
0806 16:03:41.929090 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:41.929166 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:33911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 16:03:41.930155 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:41.929848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:33911 (local address 127.0.0.1:32996)
0806 16:03:41.929993 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:41.930064 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:33911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 16:03:41.930377 16709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.189:33911: Network error: Client connection negotiation failed: client connection to 127.25.31.189:33911: connect: Connection refused (error 111)
I20260806 16:03:41.998617 16709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } attempt: 2
I20260806 16:03:41.999516 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:41.999177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:33911 (local address 127.0.0.1:33000)
0806 16:03:41.999346 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:41.999423 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:33911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 16:03:41.999583 16716 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:03:41.999939 16716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:41.999998 16716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:42.000106 16716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 16:03:42.000379 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.000161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:33911 (local address 127.0.0.1:33012)
0806 16:03:42.000263 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.000318 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.189:33911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 16:03:42.000519 16709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.189:33911: Network error: Client connection negotiation failed: client connection to 127.25.31.189:33911: connect: Connection refused (error 111)
W20260806 16:03:42.004293 16716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:42.004364 16716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:42.004401 16716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:42.004460 16716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:42.004494 16716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:42.013526 16716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:38717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:34563,127.25.31.189:33911,127.25.31.188:35493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.189.keytab
--principal=kudu/127.25.31.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.189:33911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.31.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:42.014911 16716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:42.015965 16716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:42.024442 16721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:42.024479 16724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:42.024668 16716 server_base.cc:1034] running on GCE node
W20260806 16:03:42.024835 16722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:42.025453 16716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:42.026628 16716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:42.027813 16716 hybrid_clock.cc:648] HybridClock initialized: now 1786032222027788 us; error 42 us; skew 500 ppm
Aug 06 16:03:42 dist-test-slave-tm1g krb5kdc[16628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032222, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:42.030603 16716 init.cc:377] Logged in from keytab as kudu/127.25.31.189@KRBTEST.COM (short username kudu)
I20260806 16:03:42.031466 16716 webserver.cc:460] Webserver started at http://127.25.31.189:43879/ using document root <none> and password file <none>
I20260806 16:03:42.032034 16716 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:42.032097 16716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:42.032258 16716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:42.034262 16716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/data/instance:
uuid: "e1845e43747143ebb07760ce71f90800"
format_stamp: "Formatted at 2026-08-06 16:03:42 on dist-test-slave-tm1g"
server_key: "0eaec9b1cee662f9b2aed4d49977d8a0"
server_key_iv: "fb6eb9d4a32bbd9ffd1e96dd3d38cbe7"
server_key_version: "encryptionkey@0"
I20260806 16:03:42.034852 16716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/wal/instance:
uuid: "e1845e43747143ebb07760ce71f90800"
format_stamp: "Formatted at 2026-08-06 16:03:42 on dist-test-slave-tm1g"
server_key: "0eaec9b1cee662f9b2aed4d49977d8a0"
server_key_iv: "fb6eb9d4a32bbd9ffd1e96dd3d38cbe7"
server_key_version: "encryptionkey@0"
I20260806 16:03:42.038354 16716 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 16:03:42.040781 16731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:42.041947 16716 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:03:42.042102 16716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/wal
uuid: "e1845e43747143ebb07760ce71f90800"
format_stamp: "Formatted at 2026-08-06 16:03:42 on dist-test-slave-tm1g"
server_key: "0eaec9b1cee662f9b2aed4d49977d8a0"
server_key_iv: "fb6eb9d4a32bbd9ffd1e96dd3d38cbe7"
server_key_version: "encryptionkey@0"
I20260806 16:03:42.042213 16716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:42.057147 16716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:42.059979 16716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:42.060176 16716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:42.066901 16716 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.189:33911
I20260806 16:03:42.067169 16785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.189:33911 every 8 connection(s)
I20260806 16:03:42.067845 16716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/data/info.pb
I20260806 16:03:42.070541 16786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:42.074060 16786 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:03:42.075104 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 16716
I20260806 16:03:42.075352 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-1/wal/instance
I20260806 16:03:42.075654 25726 external_mini_cluster.cc:1428] Setting key 2484e39be4cc48d39884fefeb35df28a
WARNING: no policy specified for kudu/127.25.31.188@KRBTEST.COM; defaulting to no policy
Aug 06 16:03:42 dist-test-slave-tm1g krb5kdc[16628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032222, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.189@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.31.188@KRBTEST.COM" created.
I20260806 16:03:42.089761 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.075944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33880 (local address 127.25.31.190:34563)
0806 16:03:42.076100 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.076105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.077000 (+ 895us) server_negotiation.cc:407] Connection header received
0806 16:03:42.078025 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.078029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.078089 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.078175 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.079782 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.080400 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.082250 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.082566 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.082647 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.085415 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:42.085418 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:42.085420 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:42.085444 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:42.087671 (+ 2227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.088139 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.088146 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.088150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.088243 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.088568 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.088574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.088579 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.088832 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:42.088910 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:42.089534 (+ 624us) server_negotiation.cc:299] Negotiation successful
0806 16:03:42.089637 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 16:03:42.090667 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.076321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:34563 (local address 127.0.0.1:33880)
0806 16:03:42.076820 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.076879 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:42.077825 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:42.078398 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:42.078422 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:42.078922 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:42.079604 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.079626 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.080549 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.080555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.082123 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.082136 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.082721 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.082727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.082963 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.083592 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:42.083618 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:42.085249 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:42.087784 (+ 2535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.087788 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.087798 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.088041 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.088337 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.088339 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.088340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.088468 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.088939 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:42.088947 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:42.089272 (+ 325us) client_negotiation.cc:769] Sending connection context
0806 16:03:42.089489 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 16:03:42.089853 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":111,"threads_started":1}
I20260806 16:03:42.091674 16709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } attempt: 3
I20260806 16:03:42.092222 16786 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:03:42 dist-test-slave-tm1g krb5kdc[16628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032222, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.189@KRBTEST.COM for kudu/127.25.31.189@KRBTEST.COM
Aug 06 16:03:42 dist-test-slave-tm1g krb5kdc[16628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032221, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for kudu/127.25.31.189@KRBTEST.COM
I20260806 16:03:42.103760 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.092749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:33911 (local address 127.0.0.1:33026)
0806 16:03:42.092837 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.092850 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:42.092919 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:42.093303 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:42.093305 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:42.093464 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:42.093579 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.093586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.094683 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.094689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.095541 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.095550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.096429 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.096433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.096524 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.097041 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:42.097052 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:42.100007 (+ 2955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:42.102020 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.102026 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.102031 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.102365 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.102667 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.102672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.102674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.102757 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.103234 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:42.103237 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:42.103320 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 16:03:42.103434 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 16:03:42.103574 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 16:03:42.103863 16792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.092866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33026 (local address 127.25.31.189:33911)
0806 16:03:42.093098 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.093107 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.093123 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:03:42.093146 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.093148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.093178 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.093680 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.093764 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.094579 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.095672 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.096301 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.096730 (+ 429us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.100143 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:42.100153 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:42.100167 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:42.100191 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:42.101862 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.102496 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.102500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.102501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.102573 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.102843 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.102848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.102850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.103122 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:42.103260 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:42.103537 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 16:03:42.103658 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":179,"thread_start_us":104,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.31.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.188.keytab.
Entry for principal kudu/127.25.31.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.188.keytab.
I20260806 16:03:42.105154 16791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.092475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33014 (local address 127.25.31.189:33911)
0806 16:03:42.092819 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.092834 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.092855 (+ 21us) server_negotiation.cc:407] Connection header received
0806 16:03:42.092902 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.092911 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.093120 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.093275 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.093706 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.094693 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.095726 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.096276 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.096393 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.100892 (+ 4499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:42.100895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:42.100897 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:42.100918 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:42.103194 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.103676 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.103680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.103682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.103865 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.104206 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.104209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.104211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.104405 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:42.104519 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:42.104866 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 16:03:42.104977 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":108,"server-negotiator.queue_time_us":235,"thread_start_us":135,"threads_started":1}
I20260806 16:03:42.105154 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.092092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:33911 (local address 127.0.0.1:33014)
0806 16:03:42.092193 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.092209 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:42.092280 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:42.093274 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:42.093280 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:42.093464 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:42.093582 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.093588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.094807 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.094811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.095625 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.095637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.096393 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.096397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.096519 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.097134 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:42.097151 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:42.100739 (+ 3588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:42.103346 (+ 2607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.103350 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.103354 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.103573 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.104000 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.104004 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.104006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.104096 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.104505 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:42.104509 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:42.104639 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 16:03:42.104812 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 16:03:42.104994 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 16:03:42.105857 25726 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
/tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.188:35493
--webserver_interface=127.25.31.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.188.keytab
--principal=kudu/127.25.31.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.31.148:38717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.31.190:34563,127.25.31.189:33911,127.25.31.188:35493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 16:03:42.105845 16786 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } has no permanent_uuid. Determining permanent_uuid...
I20260806 16:03:42.106613 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.106296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:35493 (local address 127.0.0.1:36826)
0806 16:03:42.106445 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.106532 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 16:03:42.106604 16709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } has no permanent_uuid. Determining permanent_uuid...
W20260806 16:03:42.107017 16732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
I20260806 16:03:42.107825 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.107616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:35493 (local address 127.0.0.1:36834)
0806 16:03:42.107716 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.107768 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 16:03:42.107975 16786 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:35493: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
I20260806 16:03:42.108146 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.107043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:35493 (local address 127.0.0.1:36832)
0806 16:03:42.107827 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.107878 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":717}
I20260806 16:03:42.108820 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.108585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:35493 (local address 127.0.0.1:36848)
0806 16:03:42.108721 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.108766 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260806 16:03:42.108964 16709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:35493: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
I20260806 16:03:42.140143 16709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } attempt: 1
I20260806 16:03:42.141028 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.140686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:35493 (local address 127.0.0.1:36854)
0806 16:03:42.140858 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.140927 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 16:03:42.141744 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.141524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:35493 (local address 127.0.0.1:36858)
0806 16:03:42.141630 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.141681 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 16:03:42.141881 16709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:35493: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
I20260806 16:03:42.157285 16786 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } attempt: 1
I20260806 16:03:42.158082 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.157815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:35493 (local address 127.0.0.1:36862)
0806 16:03:42.157956 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.158012 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 16:03:42.158859 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.158674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:35493 (local address 127.0.0.1:36876)
0806 16:03:42.158766 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.158809 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 16:03:42.159061 16786 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:35493: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
W20260806 16:03:42.216162 16793 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 16:03:42.216472 16793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 16:03:42.216532 16793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 16:03:42.216609 16793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 16:03:42.217028 16709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } attempt: 2
I20260806 16:03:42.217855 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.217530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:35493 (local address 127.0.0.1:36884)
0806 16:03:42.217693 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.217761 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 16:03:42.218616 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.218410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:35493 (local address 127.0.0.1:36896)
0806 16:03:42.218506 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.218554 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 16:03:42.218744 16709 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:35493: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
W20260806 16:03:42.221168 16793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 16:03:42.221216 16793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 16:03:42.221235 16793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 16:03:42.221292 16793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 16:03:42.221314 16793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 16:03:42.226477 16793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.31.148:38717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.31.190:34563,127.25.31.189:33911,127.25.31.188:35493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/krb5kdc/kudu_127.25.31.188.keytab
--principal=kudu/127.25.31.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.31.188:35493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.31.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 15:06:40 UTC on ccycloud.quasar-amrkwm.root.comops.site
build id 4570627
I20260806 16:03:42.227202 16786 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } attempt: 2
I20260806 16:03:42.227531 16793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 16:03:42.227885 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.227630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:35493 (local address 127.0.0.1:36910)
0806 16:03:42.227765 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.227817 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 16:03:42.228603 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.228416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:35493 (local address 127.0.0.1:36920)
0806 16:03:42.228514 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.228556 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 16:03:42.228735 16793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 16:03:42.228788 16786 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.31.188:35493: Network error: Client connection negotiation failed: client connection to 127.25.31.188:35493: connect: Connection refused (error 111)
W20260806 16:03:42.236935 16802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 16:03:42.236945 16800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:42.237066 16793 server_base.cc:1034] running on GCE node
W20260806 16:03:42.236945 16799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 16:03:42.237762 16793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 16:03:42.238834 16793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 16:03:42.239993 16793 hybrid_clock.cc:648] HybridClock initialized: now 1786032222239981 us; error 21 us; skew 500 ppm
Aug 06 16:03:42 dist-test-slave-tm1g krb5kdc[16628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032222, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 16:03:42.242982 16793 init.cc:377] Logged in from keytab as kudu/127.25.31.188@KRBTEST.COM (short username kudu)
I20260806 16:03:42.243778 16793 webserver.cc:460] Webserver started at http://127.25.31.188:41751/ using document root <none> and password file <none>
I20260806 16:03:42.244300 16793 fs_manager.cc:359] Metadata directory not provided
I20260806 16:03:42.244359 16793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 16:03:42.244593 16793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 16:03:42.246660 16793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/data/instance:
uuid: "92ae36da80af48e2bf4c32d72d9cfea9"
format_stamp: "Formatted at 2026-08-06 16:03:42 on dist-test-slave-tm1g"
server_key: "306f7ac57837cdec5a6cfff8e15a498e"
server_key_iv: "c1dcdf3b226b4caecfd782a530d9903a"
server_key_version: "encryptionkey@0"
I20260806 16:03:42.247216 16793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/wal/instance:
uuid: "92ae36da80af48e2bf4c32d72d9cfea9"
format_stamp: "Formatted at 2026-08-06 16:03:42 on dist-test-slave-tm1g"
server_key: "306f7ac57837cdec5a6cfff8e15a498e"
server_key_iv: "c1dcdf3b226b4caecfd782a530d9903a"
server_key_version: "encryptionkey@0"
I20260806 16:03:42.250732 16793 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 16:03:42.252884 16809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 16:03:42.254014 16793 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 16:03:42.254134 16793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/data,/tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/wal
uuid: "92ae36da80af48e2bf4c32d72d9cfea9"
format_stamp: "Formatted at 2026-08-06 16:03:42 on dist-test-slave-tm1g"
server_key: "306f7ac57837cdec5a6cfff8e15a498e"
server_key_iv: "c1dcdf3b226b4caecfd782a530d9903a"
server_key_version: "encryptionkey@0"
I20260806 16:03:42.254231 16793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 16:03:42.279548 16793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 16:03:42.282274 16793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 16:03:42.282507 16793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 16:03:42.289125 16793 rpc_server.cc:307] RPC server started. Bound to: 127.25.31.188:35493
I20260806 16:03:42.289408 16862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.31.188:35493 every 8 connection(s)
I20260806 16:03:42.290086 16793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/data/info.pb
I20260806 16:03:42.292639 25726 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu as pid 16793
I20260806 16:03:42.292773 25726 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJ2c_Cg/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786031939584994-25726-0/minicluster-data/master-2/wal/instance
I20260806 16:03:42.292951 16863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 16:03:42.293123 25726 external_mini_cluster.cc:1428] Setting key 1a4550ef521de7c67046d5d2cb7063a4
I20260806 16:03:42.297859 16863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:03:42 dist-test-slave-tm1g krb5kdc[16628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.188@KRBTEST.COM
Aug 06 16:03:42 dist-test-slave-tm1g krb5kdc[16628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032222, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.188@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
I20260806 16:03:42.307839 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.294489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36936 (local address 127.25.31.188:35493)
0806 16:03:42.294874 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.294892 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.294936 (+ 44us) server_negotiation.cc:407] Connection header received
0806 16:03:42.295676 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.295695 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.296019 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.296288 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.297215 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.298239 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.299423 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.300101 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.300244 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.302975 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:42.302980 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:42.302988 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:42.303014 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:42.305034 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.305545 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.305552 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.305558 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.305684 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.305994 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.305996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.305997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.306406 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:42.306613 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:42.306894 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 16:03:42.307150 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":129,"threads_started":1}
I20260806 16:03:42.309361 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.299748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33890 (local address 127.25.31.190:34563)
0806 16:03:42.299881 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.299888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.300161 (+ 273us) server_negotiation.cc:407] Connection header received
0806 16:03:42.300332 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.300335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.300380 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.300472 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.301252 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.301739 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.303404 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.303666 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.303744 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.306164 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:42.306167 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:42.306169 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:42.306191 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:42.307618 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.308125 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.308129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.308131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.308226 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.308483 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.308486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.308488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.308692 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:42.308792 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:42.309158 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 16:03:42.309255 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 16:03:42.309412 16867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.299658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.190:34563 (local address 127.0.0.1:33890)
0806 16:03:42.300044 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.300081 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:42.300210 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:42.300517 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:42.300523 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:42.300996 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:42.301125 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.301130 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.302476 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.302481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.303285 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.303294 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.303785 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.303792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.303894 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.304293 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:42.304312 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:42.306025 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:42.307741 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.307750 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.307755 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.308020 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.308321 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.308323 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.308324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.308396 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.308794 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:42.308799 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:42.308930 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 16:03:42.309079 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 16:03:42.309235 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":102,"threads_started":1}
I20260806 16:03:42.310508 16863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:03:42 dist-test-slave-tm1g krb5kdc[16628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.190@KRBTEST.COM
Aug 06 16:03:42 dist-test-slave-tm1g krb5kdc[16628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032222, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.188@KRBTEST.COM for kudu/127.25.31.189@KRBTEST.COM
I20260806 16:03:42.317903 16709 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } attempt: 3
I20260806 16:03:42.319483 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.310326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33896 (local address 127.25.31.190:34563)
0806 16:03:42.310428 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.310431 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.310455 (+ 24us) server_negotiation.cc:407] Connection header received
0806 16:03:42.310580 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.310582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.310640 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.310718 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.311425 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.311977 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.312763 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.312962 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.313033 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.315657 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:42.315660 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:42.315661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:42.315703 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:42.317175 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.317629 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.317631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.317633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.317699 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.318089 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.318093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.318095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.318500 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:42.318716 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:42.319076 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 16:03:42.319205 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 16:03:42.320029 16867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.310947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.189:33911 (local address 127.0.0.1:33038)
0806 16:03:42.311050 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.311065 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:42.311133 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:42.311535 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:42.311537 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:42.311657 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:42.311736 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.311741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.312575 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.312578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.313237 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.313249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.313826 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.313831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.313975 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.314365 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:42.314379 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:42.315980 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:42.317942 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.317945 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.317947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.318236 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.318627 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.318630 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.318631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.318716 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.319552 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:42.319555 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:42.319649 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 16:03:42.319756 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 16:03:42.319876 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 16:03:42.319993 16786 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } attempt: 3
I20260806 16:03:42.321342 16791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.311104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33038 (local address 127.25.31.189:33911)
0806 16:03:42.311233 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.311238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.311252 (+ 14us) server_negotiation.cc:407] Connection header received
0806 16:03:42.311295 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.311298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.311347 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.311426 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.311863 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.312467 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.313367 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.313708 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.313790 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.316112 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:42.316115 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:42.316117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:42.316141 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:42.317797 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.318393 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.318399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.318401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.318505 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.319246 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.319250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.319251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.319446 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:42.319558 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:42.319814 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 16:03:42.319928 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 16:03:42.322184 16863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 16:03:42 dist-test-slave-tm1g krb5kdc[16628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032221, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.190@KRBTEST.COM for kudu/127.25.31.188@KRBTEST.COM
Aug 06 16:03:42 dist-test-slave-tm1g krb5kdc[16628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.31.189@KRBTEST.COM
I20260806 16:03:42.327373 16711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.318361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:35493 (local address 127.0.0.1:36946)
0806 16:03:42.318523 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.318538 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:42.318602 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:42.318816 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:42.318818 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:42.319011 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:42.319106 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.319115 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.320066 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.320072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.320849 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.320859 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.321624 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.321628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.321710 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.322169 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:42.322183 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:42.323563 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:42.325824 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.325827 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.325829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.326024 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.326344 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.326350 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.326353 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.326441 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.326857 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:42.326860 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:42.326953 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 16:03:42.327058 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 16:03:42.327231 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260806 16:03:42.327371 16866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.318447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36946 (local address 127.25.31.188:35493)
0806 16:03:42.318578 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.318584 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.318599 (+ 15us) server_negotiation.cc:407] Connection header received
0806 16:03:42.318671 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.318676 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.318714 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.318788 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.319231 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.319963 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.320963 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.321333 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.321439 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.323698 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:42.323702 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:42.323707 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:42.323730 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:42.325704 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.326163 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.326166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.326167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.326233 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.326535 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.326538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.326540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.326711 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:42.326816 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:42.327123 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 16:03:42.327230 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 16:03:42 dist-test-slave-tm1g krb5kdc[16628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032222, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.189@KRBTEST.COM for kudu/127.25.31.188@KRBTEST.COM
Aug 06 16:03:42 dist-test-slave-tm1g krb5kdc[16628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786032222, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.31.188@KRBTEST.COM for kudu/127.25.31.188@KRBTEST.COM
I20260806 16:03:42.330977 16709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176: Bootstrap starting.
I20260806 16:03:42.331643 16792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.320936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33046 (local address 127.25.31.189:33911)
0806 16:03:42.321034 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.321037 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.321075 (+ 38us) server_negotiation.cc:407] Connection header received
0806 16:03:42.321218 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.321221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.321262 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.321335 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.322177 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.322950 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.324056 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.324418 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.324502 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.327418 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:42.327422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:42.327424 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:42.327448 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:42.329337 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.330019 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.330022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.330024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.330110 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.330452 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.330455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.330457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.330664 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:42.330879 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:42.331441 (+ 562us) server_negotiation.cc:299] Negotiation successful
0806 16:03:42.331544 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 16:03:42.333947 16709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:42.334219 16867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.322633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:35493 (local address 127.0.0.1:36974)
0806 16:03:42.323409 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.323423 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:42.323505 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:42.323893 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:42.323897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:42.324061 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:42.324172 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.324184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.325486 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.325490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.326469 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.326484 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.327777 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.327780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.328283 (+ 503us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.329181 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:42.329198 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:42.330686 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:42.332675 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.332679 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.332684 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.332883 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.333153 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.333156 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.333157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.333258 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.333708 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:42.333711 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:42.333805 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 16:03:42.333923 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 16:03:42.334055 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":409}
I20260806 16:03:42.334280 16869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.322905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36974 (local address 127.25.31.188:35493)
0806 16:03:42.323691 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.323699 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.323709 (+ 10us) server_negotiation.cc:407] Connection header received
0806 16:03:42.323735 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.323738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.323778 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.324280 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.324423 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.325362 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.326604 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.326974 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.327772 (+ 798us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.330817 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:42.330820 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:42.330822 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:42.330849 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:42.332565 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.332979 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.332985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.332987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.333071 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.333365 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.333368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.333370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.333598 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:42.333702 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:42.333982 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 16:03:42.334111 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":728,"thread_start_us":98,"threads_started":1}
I20260806 16:03:42.334893 25726 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 16:03:42.335470 16709 log.cc:824] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:42.337448 16863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9: Bootstrap starting.
I20260806 16:03:42.337616 16709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176: No bootstrap required, opened a new log
I20260806 16:03:42.337924 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.320597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.31.188:35493 (local address 127.0.0.1:36960)
0806 16:03:42.320689 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 16:03:42.320705 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 16:03:42.320776 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 16:03:42.322579 (+ 1803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 16:03:42.322582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 16:03:42.322743 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 16:03:42.322823 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.322829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.324331 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.324335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.325112 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 16:03:42.325124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.327335 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.327338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 16:03:42.327430 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.327842 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 16:03:42.327858 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 16:03:42.334335 (+ 6477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 16:03:42.336346 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.336350 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.336352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.336608 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.336926 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 16:03:42.336930 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 16:03:42.336931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 16:03:42.337024 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 16:03:42.337440 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 16:03:42.337443 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 16:03:42.337544 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 16:03:42.337650 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 16:03:42.337775 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 16:03:42.338003 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.321045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36960 (local address 127.25.31.188:35493)
0806 16:03:42.322352 (+ 1307us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.322360 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.322372 (+ 12us) server_negotiation.cc:407] Connection header received
0806 16:03:42.322404 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.322406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.322447 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.322939 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.323039 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.324222 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.325266 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.327201 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.327631 (+ 430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.334456 (+ 6825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:42.334461 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:42.334465 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:42.334489 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:42.336224 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.336725 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.336728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.336730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.336817 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.337128 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.337131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.337133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.337330 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:42.337428 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:42.337711 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 16:03:42.337818 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":1223,"thread_start_us":97,"threads_started":1}
I20260806 16:03:42.339947 16863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:42.340674 16786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800: Bootstrap starting.
I20260806 16:03:42.340775 16863 log.cc:824] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:42.340700 16709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } }
I20260806 16:03:42.340862 16709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dbcb460d1ce41a49a474b474c456176, State: Initialized, Role: FOLLOWER
I20260806 16:03:42.341204 16709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } }
I20260806 16:03:42.341534 16683 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8dbcb460d1ce41a49a474b474c456176"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33896
I20260806 16:03:42.342574 16709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:42.342886 16683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 16:03:42.343005 16683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:42.343039 16863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9: No bootstrap required, opened a new log
I20260806 16:03:42.343114 16786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800: Neither blocks nor log segments found. Creating new log.
I20260806 16:03:42.343858 16786 log.cc:824] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800: Log is configured to *not* fsync() on all Append() calls
I20260806 16:03:42.344324 16683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } }
I20260806 16:03:42.345268 16683 leader_election.cc:288] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [CANDIDATE]: Term 1 election: Requested vote from peers e1845e43747143ebb07760ce71f90800 (127.25.31.189:33911), 92ae36da80af48e2bf4c32d72d9cfea9 (127.25.31.188:35493)
I20260806 16:03:42.345733 16872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } } }
I20260806 16:03:42.345927 16872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 16:03:42.346028 16662 tablet.cc:2369] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 16:03:42.348207 16884 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:03:42.348297 16884 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:03:42.349332 16863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } }
I20260806 16:03:42.349570 16863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ae36da80af48e2bf4c32d72d9cfea9, State: Initialized, Role: FOLLOWER
I20260806 16:03:42.349905 16863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } }
I20260806 16:03:42.350553 16892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } } }
I20260806 16:03:42.350689 16892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:03:42.351617 16863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:42.349792 16641 master_runner.cc:186] Error getting master registration for 127.25.31.189:33911: OK, instance_id { permanent_uuid: "e1845e43747143ebb07760ce71f90800" instance_seqno: 1786032222065063 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:03:42.353260 16641 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:03:42.353386 16836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8dbcb460d1ce41a49a474b474c456176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "92ae36da80af48e2bf4c32d72d9cfea9"
I20260806 16:03:42.353611 16836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:42.355024 16836 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dbcb460d1ce41a49a474b474c456176 in term 1.
I20260806 16:03:42.356266 16657 leader_election.cc:302] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dbcb460d1ce41a49a474b474c456176, 92ae36da80af48e2bf4c32d72d9cfea9; no voters:
I20260806 16:03:42.357227 16872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 16:03:42.357457 16872 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [term 1 LEADER]: Becoming Leader. State: Replica: 8dbcb460d1ce41a49a474b474c456176, State: Running, Role: LEADER
I20260806 16:03:42.357788 16872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } }
W20260806 16:03:42.359014 16904 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:03:42.359081 16904 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:03:42.359558 16872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dbcb460d1ce41a49a474b474c456176. Latest consensus state: current_term: 1 leader_uuid: "8dbcb460d1ce41a49a474b474c456176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } } }
I20260806 16:03:42.359674 16872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:42.360553 16786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800: No bootstrap required, opened a new log
I20260806 16:03:42.363593 16792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.349115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33062 (local address 127.25.31.189:33911)
0806 16:03:42.349228 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.349231 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.349749 (+ 518us) server_negotiation.cc:407] Connection header received
0806 16:03:42.349937 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.349940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.349986 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.350038 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.351237 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.351798 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.352808 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.353045 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.353151 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.354921 (+ 1770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:42.354924 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:42.354927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:42.354961 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:42.356632 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.362176 (+ 5544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.362180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.362181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.362272 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.362618 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.362621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.362623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.362833 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:42.362947 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:42.363288 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 16:03:42.363415 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 16:03:42.364082 16908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 16:03:42.364759 16786 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } }
I20260806 16:03:42.365043 16786 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1845e43747143ebb07760ce71f90800, State: Initialized, Role: FOLLOWER
I20260806 16:03:42.365433 16786 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } }
I20260806 16:03:42.362509 16793 master_runner.cc:186] Error getting master registration for 127.25.31.189:33911: OK, instance_id { permanent_uuid: "e1845e43747143ebb07760ce71f90800" instance_seqno: 1786032222065063 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 16:03:42.366122 16793 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 16:03:42.366851 16786 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 16:03:42.366904 16758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8dbcb460d1ce41a49a474b474c456176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e1845e43747143ebb07760ce71f90800"
I20260806 16:03:42.367166 16758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800 [term 0 FOLLOWER]: Advancing to term 1
I20260806 16:03:42.368604 16758 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8dbcb460d1ce41a49a474b474c456176 in term 1.
I20260806 16:03:42.371187 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.349775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33904 (local address 127.25.31.190:34563)
0806 16:03:42.349893 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.349900 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.359391 (+ 9491us) server_negotiation.cc:407] Connection header received
0806 16:03:42.359572 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.359575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.359623 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.359672 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.360531 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.361267 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.362917 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.363249 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.363324 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.367007 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:42.367010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:42.367013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:42.367040 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:42.368832 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.369799 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.369802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.369804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.369905 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.370217 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.370220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.370222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.370470 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:42.370583 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:42.370926 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 16:03:42.371043 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 16:03:42.372051 16908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260806 16:03:42.372046 16920 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 16:03:42.372112 16920 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 16:03:42.372562 16909 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } } }
I20260806 16:03:42.372676 16909 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:03:42.374891 16869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:42.350483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36986 (local address 127.25.31.188:35493)
0806 16:03:42.350598 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.350604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.364731 (+ 14127us) server_negotiation.cc:407] Connection header received
0806 16:03:42.364891 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.364894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.364941 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.364993 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.366067 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.366910 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.368778 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:42.369148 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:42.369224 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:42.370872 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 16:03:42.370876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 16:03:42.370880 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 16:03:42.370911 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 16:03:42.372933 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.373468 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.373472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.373473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.373564 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 16:03:42.373880 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 16:03:42.373883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 16:03:42.373884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 16:03:42.374103 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 16:03:42.374225 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:42.374626 (+ 401us) server_negotiation.cc:299] Negotiation successful
0806 16:03:42.374751 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 16:03:42.378147 16836 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9 [term 1 FOLLOWER]: Refusing update from remote peer 8dbcb460d1ce41a49a474b474c456176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:42.378250 16758 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800 [term 1 FOLLOWER]: Refusing update from remote peer 8dbcb460d1ce41a49a474b474c456176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 16:03:42.379216 16872 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:03:42.379648 16872 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 16:03:42.383109 16892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dbcb460d1ce41a49a474b474c456176. Latest consensus state: current_term: 1 leader_uuid: "8dbcb460d1ce41a49a474b474c456176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } } }
I20260806 16:03:42.383224 16892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:03:42.383267 16909 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dbcb460d1ce41a49a474b474c456176. Latest consensus state: current_term: 1 leader_uuid: "8dbcb460d1ce41a49a474b474c456176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } } }
I20260806 16:03:42.383414 16909 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:03:42.386107 16906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8dbcb460d1ce41a49a474b474c456176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } } }
I20260806 16:03:42.386234 16906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:42.387247 16892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8dbcb460d1ce41a49a474b474c456176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } } }
I20260806 16:03:42.387351 16892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:03:42.388411 16908 catalog_manager.cc:1347] Generated new cluster ID: acddd740b83246c89ffd171c7bbd5a7a
I20260806 16:03:42.388481 16908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 16:03:42.398084 16908 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 16:03:42.399281 16908 catalog_manager.cc:1504] Loading token signing keys...
I20260806 16:03:42.400080 16906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8dbcb460d1ce41a49a474b474c456176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } } }
I20260806 16:03:42.400203 16906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176 [sys.catalog]: This master's current role is: LEADER
I20260806 16:03:42.402141 16909 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8dbcb460d1ce41a49a474b474c456176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8dbcb460d1ce41a49a474b474c456176" member_type: VOTER last_known_addr { host: "127.25.31.190" port: 34563 } } peers { permanent_uuid: "e1845e43747143ebb07760ce71f90800" member_type: VOTER last_known_addr { host: "127.25.31.189" port: 33911 } } peers { permanent_uuid: "92ae36da80af48e2bf4c32d72d9cfea9" member_type: VOTER last_known_addr { host: "127.25.31.188" port: 35493 } } }
I20260806 16:03:42.402251 16909 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 16:03:42.410432 16908 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8dbcb460d1ce41a49a474b474c456176: Generated new TSK 0
I20260806 16:03:42.411152 16908 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 16:03:42.441920 16792 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:42.439368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33074 (local address 127.25.31.189:33911)
0806 16:03:42.439497 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.439502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.440785 (+ 1283us) server_negotiation.cc:407] Connection header received
0806 16:03:42.440995 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.440999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.441049 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.441109 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.441754 (+ 645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33074: BlockingRecv error: recv got EOF from 127.0.0.1:33074 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 16:03:42.443457 16712 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:42.440180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33912 (local address 127.25.31.190:34563)
0806 16:03:42.440294 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.440298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.442574 (+ 2276us) server_negotiation.cc:407] Connection header received
0806 16:03:42.442743 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.442747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.442793 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.442854 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.443337 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33912: BlockingRecv error: recv got EOF from 127.0.0.1:33912 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 16:03:42.445627 16869 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:42.444208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36998 (local address 127.25.31.188:35493)
0806 16:03:42.444318 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.444322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.444607 (+ 285us) server_negotiation.cc:407] Connection header received
0806 16:03:42.444774 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.444779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.444828 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.444896 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.445511 (+ 615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36998: BlockingRecv error: recv got EOF from 127.0.0.1:36998 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 16:03:42.449982 16792 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:42.448485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33082 (local address 127.25.31.189:33911)
0806 16:03:42.448582 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.448586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.449068 (+ 482us) server_negotiation.cc:407] Connection header received
0806 16:03:42.449277 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.449280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.449358 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.449410 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.449889 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33082: BlockingRecv error: recv got EOF from 127.0.0.1:33082 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 16:03:42.455041 16792 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:42.453476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33088 (local address 127.25.31.189:33911)
0806 16:03:42.453568 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.453572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.454155 (+ 583us) server_negotiation.cc:407] Connection header received
0806 16:03:42.454362 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.454365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.454414 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.454463 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.454955 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33088: BlockingRecv error: recv got EOF from 127.0.0.1:33088 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 16:03:42.460474 16792 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:42.459077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33092 (local address 127.25.31.189:33911)
0806 16:03:42.459164 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.459168 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.459669 (+ 501us) server_negotiation.cc:407] Connection header received
0806 16:03:42.459835 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.459838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.459896 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.459943 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.460384 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33092: BlockingRecv error: recv got EOF from 127.0.0.1:33092 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 16:03:42.467752 16792 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:42.466362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33094 (local address 127.25.31.189:33911)
0806 16:03:42.466485 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.466488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.466972 (+ 484us) server_negotiation.cc:407] Connection header received
0806 16:03:42.467137 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.467140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.467185 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.467231 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.467670 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33094: BlockingRecv error: recv got EOF from 127.0.0.1:33094 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 16:03:42.479123 16792 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:42.477631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33102 (local address 127.25.31.189:33911)
0806 16:03:42.477719 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.477723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.478245 (+ 522us) server_negotiation.cc:407] Connection header received
0806 16:03:42.478476 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.478480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.478527 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.478574 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.479038 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33102: BlockingRecv error: recv got EOF from 127.0.0.1:33102 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 16:03:42.499012 16792 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:42.497304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33110 (local address 127.25.31.189:33911)
0806 16:03:42.497436 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.497443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.498001 (+ 558us) server_negotiation.cc:407] Connection header received
0806 16:03:42.498193 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.498196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.498247 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.498335 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.498885 (+ 550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33110: BlockingRecv error: recv got EOF from 127.0.0.1:33110 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 16:03:42.536294 16792 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:42.533621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33118 (local address 127.25.31.189:33911)
0806 16:03:42.533775 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.533780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.535301 (+ 1521us) server_negotiation.cc:407] Connection header received
0806 16:03:42.535519 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.535523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.535576 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.535628 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.536175 (+ 547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33118: BlockingRecv error: recv got EOF from 127.0.0.1:33118 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 16:03:42.604221 16792 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:42.602943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33128 (local address 127.25.31.189:33911)
0806 16:03:42.603101 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.603106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.603243 (+ 137us) server_negotiation.cc:407] Connection header received
0806 16:03:42.603440 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.603444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.603504 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.603601 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.604072 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33128: BlockingRecv error: recv got EOF from 127.0.0.1:33128 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260806 16:03:42.735780 16792 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:42.734460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33130 (local address 127.25.31.189:33911)
0806 16:03:42.734611 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.734616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.734854 (+ 238us) server_negotiation.cc:407] Connection header received
0806 16:03:42.735026 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.735029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.735079 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.735173 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.735671 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33130: BlockingRecv error: recv got EOF from 127.0.0.1:33130 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 16:03:42.995373 16792 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 16:03:42.994178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33134 (local address 127.25.31.189:33911)
0806 16:03:42.994344 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:42.994348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:42.994462 (+ 114us) server_negotiation.cc:407] Connection header received
0806 16:03:42.994579 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:42.994582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:42.994633 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:42.994754 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 16:03:42.995247 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33134: BlockingRecv error: recv got EOF from 127.0.0.1:33134 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 16:03:43.360348 16904 catalog_manager.cc:1259] Loaded cluster ID: acddd740b83246c89ffd171c7bbd5a7a
I20260806 16:03:43.360441 16904 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9: loading cluster ID for follower catalog manager: success
I20260806 16:03:43.362196 16904 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9: acquiring CA information for follower catalog manager: success
I20260806 16:03:43.363365 16904 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 92ae36da80af48e2bf4c32d72d9cfea9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:03:43.373265 16920 catalog_manager.cc:1259] Loaded cluster ID: acddd740b83246c89ffd171c7bbd5a7a
I20260806 16:03:43.373335 16920 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800: loading cluster ID for follower catalog manager: success
I20260806 16:03:43.374929 16920 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800: acquiring CA information for follower catalog manager: success
I20260806 16:03:43.375981 16920 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e1845e43747143ebb07760ce71f90800: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 16:03:43.513641 16997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:43.509648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33146 (local address 127.25.31.189:33911)
0806 16:03:43.509930 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:43.509934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:43.509950 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:03:43.510083 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:43.510085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:43.510214 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:43.510345 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:43.510638 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:43.511170 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:43.512022 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:43.512239 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:43.512315 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:43.512686 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:43.513132 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:43.513233 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:43.513371 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 16:03:43.513427 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":75,"threads_started":1}
I20260806 16:03:43.520412 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:43.516726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37008 (local address 127.25.31.188:35493)
0806 16:03:43.516969 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:43.516976 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:43.516993 (+ 17us) server_negotiation.cc:407] Connection header received
0806 16:03:43.517092 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:43.517094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:43.517204 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:43.517325 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:43.517567 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:43.518086 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:43.518983 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:43.519208 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:43.519317 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:43.519620 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:43.520016 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:43.520140 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:43.520209 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 16:03:43.520262 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":56,"threads_started":1}
I20260806 16:03:43.527420 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:43.523651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37018 (local address 127.25.31.188:35493)
0806 16:03:43.523751 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:43.523755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:43.523859 (+ 104us) server_negotiation.cc:407] Connection header received
0806 16:03:43.523959 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:43.523965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:43.524023 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:43.524082 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:43.524453 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:43.525185 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:43.526074 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:43.526421 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:43.526544 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:43.526788 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:43.526858 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:43.526937 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:43.527243 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 16:03:43.527291 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 16:03:43.527609 16997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:43.523247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33160 (local address 127.25.31.189:33911)
0806 16:03:43.523384 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:43.523390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:43.523740 (+ 350us) server_negotiation.cc:407] Connection header received
0806 16:03:43.523956 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:43.523961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:43.524036 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:43.524084 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:43.524440 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:43.524938 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:43.526154 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:43.526533 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:43.526630 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:43.527047 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:43.527127 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:43.527196 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:43.527399 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 16:03:43.527452 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 16:03:43.528028 17010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 16:03:43.523189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33924 (local address 127.25.31.190:34563)
0806 16:03:43.523501 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 16:03:43.523506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 16:03:43.523522 (+ 16us) server_negotiation.cc:407] Connection header received
0806 16:03:43.523737 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 16:03:43.523741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 16:03:43.523919 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 16:03:43.524038 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 16:03:43.524280 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:43.525124 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:43.526270 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 16:03:43.526690 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 16:03:43.526785 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 16:03:43.527206 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 16:03:43.527506 (+ 300us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 16:03:43.527600 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 16:03:43.527809 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 16:03:43.527881 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":74,"threads_started":1}
I20260806 16:03:43.531378 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 16641
I20260806 16:03:43.539055 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 16716
I20260806 16:03:43.545608 25726 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJ2c_Cg/build/debug/bin/kudu with pid 16793
2026-08-06T16:03:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3036 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6202 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (283954 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST